System and methods for encoding information for printed articles

ABSTRACT

The present disclosure relates generally to digital watermarking for spot colors. In one implementation a substitute spot color+CMY tint is selected to replace an original spot color. The CMY tint can be transformed to carry a digital watermark signal. Of course, other features, combinations and technology are described herein.

RELATED APPLICATION DATA

This application claims the benefit of U.S. Provisional PatentApplication No. 62/164,479, filed May 20, 2015, which is herebyincorporated herein by reference in its entirety.

This application is a continuation in part of U.S. patent applicationSer. No. 14/616,686, filed Feb. 7, 2015 (published as US 2015-0156369A1, now U.S. Pat. No. 9,380,186), which claims the benefit of U.S.Patent Application No. 62/102,247, filed Jan. 12, 2015, 62/063,790,filed Oct. 14, 2014, 62/063,360, filed Oct. 13, 2014, and 62/036,444,filed Aug. 12, 2014. Each of the above patent documents is herebyincorporated herein by reference in its entirety.

This application is related to International Patent Application No.PCT/US15/44904 filed Aug. 12, 2015 (published as WO 2016/025631 A1),which are each hereby incorporated herein by reference in its entirety.

This application is also related to U.S. patent application Ser. No.14/588,636, filed Jan. 2, 2015 (published as US 2015-0187039 A1, issuedas U.S. Pat. No. 9,401,001), which claims the benefit of U.S.Provisional application No. 61/923,060, filed Jan. 2, 2014. Thisapplication is also related to U.S. patent application Ser. No.13/975,919, filed Aug. 26, 2013 (issued as U.S. Pat. No. 9,449,357),which claims the benefit of U.S. Provisional Application No. 61/749,767,filed Jan. 7, 2013 and 61/693,106, filed Aug. 24, 2012. This applicationis also related to U.S. Provisional Patent Application No. 62/152,745,filed Apr. 24, 2015, and 62/136,146, filed Mar. 20, 2015. Thisapplication is also related to U.S. Pat. No. 8,199,969, US PublishedPatent Application Nos. US 2010-0150434 A1 and US 2013-0329006 A1; andU.S. Provisional Application No. 62/106,685, filed Jan. 22, 2015,62/102,547, filed Jan. 12, 2015, 61/693,106, filed Aug. 24, 2012,61/716,591, filed Oct. 21, 2012, and 61/719,920, filed Oct. 29, 2012.Each of the above patent documents is hereby incorporated herein byreference in its entirety.

Each of the above patent documents is hereby incorporated herein byreference in its entirety. Such incorporation by reference, and allfollowing incorporations by reference, are intended to incorporate theentire application including the entire specification, all drawings andany appendices, even if a patent document is only discussed with respectto a specific portion thereof.

TECHNICAL FIELD

The present disclosure relates generally to color science technology,printing technology, data hiding, color visibility models and digitalwatermarking, particularly for product packaging and other printedobjects.

BACKGROUND AND SUMMARY

The term “steganography” generally implies data hiding. One form of datahiding includes digital watermarking. For purposes of this disclosure,the terms “digital watermark,” “watermark” and “data hiding” are usedinterchangeably. We sometimes use the terms “embedding,” “embed,” anddata hiding” (and variants thereof) to mean modulating or transformingdata representing imagery or video to include information therein. Forexample, data hiding may seek to hide or embed an information signal(e.g., a plural bit payload or a modified version of such, e.g., a 2-Derror corrected, spread spectrum signal) in a host signal. This can beaccomplished, e.g., by modulating a host signal (e.g., image, video oraudio) in some fashion to carry the information signal. One way tomodulate a host signal, as described in detail herein, is to overprint afirst color with additional colors. The additional colors may carry orrepresent the information signal. We use the terms “decode,” “detect,”and “read” (and variants thereof) interchangeably to mean detecting orrecovering an embedded digital watermark.

Some of the present assignee's work in steganography, data hiding anddigital watermarking is reflected, e.g., in U.S. Pat. Nos. 6,947,571;6,912,295; 6,891,959, 6,763,123; 6,718,046; 6,614,914; 6,590,996;6,408,082; 6,122,403 and 5,862,260, and in published specifications WO9953428 and WO 0007356 (corresponding to U.S. Pat. Nos. 6,449,377 and6,345,104). Each of these patent documents is hereby incorporated byreference herein in its entirety. Of course, a great many otherapproaches are familiar to those skilled in the art. The artisan ispresumed to be familiar with a full range of literature concerningsteganography, data hiding and digital watermarking.

This disclosure focuses on data hiding with printed colors, e.g.,embedding information signals in so-called spot colors and processcolors. Of course, our techniques, methods and systems will be usefulfor other color schemes as well, e.g., digital printing.

Spot colors may include premixed inks for use instead of or in additionto process color inks. In many print environments, each spot color inktypically uses its own printing plate on a print press. Spot colors canbe used instead of or in addition to process colors for better coloraccuracy, better color consistency, and colors outside of process inkgamut and for technologies which are prone to specific printing errors.A common spot color system is PANTONE (http://www.pantone.com/). ThePANTONE system defines several hundred different inks.

Process colors can be printed using a combination of four standardprocess inks: Cyan, Magenta, Yellow and Black (CMYK). Considering thatevery color used in some printing presses uses its own plate, it ishighly impractical to print using every color in a design. Process colorprinting was developed, in part, to address this impracticality, sincemost colors can be accurately approximated with a combination of thesefour process colors, CMYK. To create a process color which includesmultiple inks, overprinting can be used.

Similar to CMYK, it is usually possible to print a percentage of a givenspot color. We refer to printing less than 100% of a spot color as“screening” (or “a screen”) the spot color or as a “spot color tint”.There are sometimes advantages to using process color equivalent tint.The process color equivalent tint can be a combination of CMYKpercentages which produce an approximation color for an original spotcolor or spot color tint. Process colors can be printed with, e.g., halftone dots.

Overprinting is the process of printing one or more colors on top ofanother in the reproduction of a design. Because of physical differencesbetween inks and substrate, the result of printing directly onto thesubstrate versus onto another ink may differ and can be considered in aprint run. In some situations, it is necessary to print the desiredcolor using a single ink or a spot color.

Various materials and techniques can be used in the printing processwhich can be considered for data hiding for spot colors and processcolors, these materials include: substrate, process colors,overprinting, spot colors, spot tint (screening) and process equivalenttints. In printing, the term “substrate” refers to a base material whicha design is printed onto. Most often, a substrate comprises paper whichcan be a variety of weights and finishes. Other common substrates incommercial printing include films, plastics, laminated plastics andfoils.

Some color science background along with our improvements and additionsare provided, below.

The color of an object is often the result of an interaction between alight source, an object and a detector (e.g., the human visual system).Other detectors include point of sale captured systems, mobile phonecameras, barcode readers, etc.

Light is radiation which can be seen, in the wavelength range of about380 to 780 nm.

Spectral reflectance can be used to describe how an object interactswith light. When reflected light is detected and interpreted through thehuman visual system it results in an object having a particular color.The most common way to capture spectral data with a device is by using aspectrophotometer.

FIG. 1(a) shows spectral reflectance of PANTONE process color inks asmeasured using an i1Pro spectrophotometer, from X-Rite Corporation,headquartered in Grand Rapids, Mich., USA. FIG. 1(a) also shows spectrumemitted by red LED illumination at or around 660 nm. FIG. 1(b) shows 931CIE 2° standard observer matching functions used for converting spectralreflectance to CIE XYZ color space.

Often color is described by artists and designers in terms of mixingpaint or inks. An artist often starts with white paper, which reflectsmost of the light. Different colored pigments are applied on top of thepaper, which reduce the amount of light reflected back. Current trendsfor printing describe subtractive four color mixing using process colorcombinations of CMYK. Yellow, for instance, reflects most of the light,it absorbs only the lower wavelengths.

In 1931, the CIE (Commission Internationale de l'Eclairage) developed away to link between wavelengths in the visible spectrum and colors whichare perceived by the human visual system. The models which the CIEdeveloped made it possible to transform color information betweenphysical responses to reflectance in color inks, illuminated displays,and capture devices such as digital cameras into a perceptually (nearly)uniform color space. The CIE XYZ color space was derived by multiplyingthe color matching functions† with the spectral power of the illuminantand the reflectance of an object, which results in a set of XYZtristimulus values for a given sample. Within the CIE model, CIE Ydescribes the luminance or perceived brightness. While the CIE X and CIEZ plane contain the chromaticities, which describes the color regardlessof luminance.

Chromaticity can be described by two parameters, hue and colorfulness.Hue or hue angle, describes the perceived color name, such as: red,green, yellow and blue. Colorfulness is the attribute which describes acolor as having more or less of its hue. A color with 0 colorfulnesswould be neutral. The CIE took the CIE XYZ space to propose apseudo-uniform color space, where calculated differences areproportional to perceptual differences between two color stimuli,formally referred to as the CIE 1976 L*a*b* (CIELAB) color space. The L*coordinate represents the perceived lightness, an L* value of 0indicates black and a value of 100 indicates white. The CIE a*coordinate position goes between “redness” (positive) and “greenness”(negative), while the CIE b* goes between “yellowness” (positive) and“blueness” (negative).

To describe how perceptually similar two colors are, the CIE developed acolor difference model, CIE ΔE₇₆. The first model developed was simplythe Euclidean distance in CIELAB between two color samples. Since then,other more complex models have been developed to address some of thenon-uniformity within the CIELAB Color-space, most notably thesensitivity to neutral or near neutral colors.

The CIELAB color difference metric is appropriate for measuring thecolor difference of a large uniform color region, however, the modeldoes not consider the spatial-color sensitivity of the human eye. Theluminance and chrominance CSF (Contrast Sensitivity Function) of thehuman visual system has been measured for various retinal illuminationlevels. The luminance CSF variation was measured by van Nes and Bouman(1967) and the chrominance CSF variation by van der Horst and Bouman(1969) and the curves are plotted in FIG. 2a for a single typicalillumination level. See, e.g., Johnson et al, “Darwinism of Color ImageDifference Models,” Proc. of IS&T/SID, 9th Color Imaging Conference,November 2001, p. 108-11, for an additional discussion of CSFs. TheJohnson et al. document is hereby incorporated herein by reference inits entirety.

A digital watermark may contain signal energy, e.g., over the spatialresolutions shown by the gray box in FIG. 2a . If the luminance andchrominance contrast sensitivity functions are integrated over this graybox region, the resultant energy ratios calculate the uniform perceptualscaling for CIELAB L*, a* and b*. Thus the watermark perceptual errorΔE_(WM) can be calculated as:ΔE _(WM)=(ΔL ²+(Δa/8)²+(Δb/16)²)^(1/2),  (1)where ΔL is the luminance variation and Δa and Δb the two chrominancevariations introduced by a watermark.

Ink overprint models predict final color obtained by overprintingseveral inks on a specific press and substrate. These models can be useddigital watermark embedding algorithm to predict (1) color of theoverprint for visibility evaluation, and (2) color of the overprint asseen by the imaging device for signal robustness evaluation. Inkoverprint models can be obtained in practice by combining two mainfactors (1) set of measured color patches printed on a real press, and(2) mathematical model interpolating the measured values while makingsome simplifying assumptions. One model can be obtained by measuring aset of color patches obtained by sampling the space of all possible inkcombinations, possibly printed multiple times and averaged. For example,for k inks and n steps of each ink, n^(k) color patches would have to beprinted and measured. This process, known as press profiling or pressfingerprinting, is often used with process inks, where a few thousandpatches are used to characterize the press. Measured values are theninterpolated and assembled into k-dimensional look-up table which isfurther consumed by software tools. ICC profiles are standardized andindustry-accepted form of such look-up tables converting k inkpercentages into either CIE XYZ or CIELAB space. For process inks,4-channel CMYK profiles are standardized to maintain consistency betweendifferent printers. For example, the GRACoL (“General Requirements forApplications in Commercial Offset Lithography”) specification includesCMYK ICC profiles recommended for commercial offset lithography.Unfortunately, full color spectral data is often not available asstandardization is still in progress. This methodology quickly becomesimpractical as spot colors are introduced due to exponential increase ofthe number of patches used to print and large number of spot colorsavailable. A previous mathematical model for ink overprint was describedby Neugebauer. For example, see, e.g., Wyble et al., “A critical reviewof spectral models applied to binary color printing,” Color Research &Application, 25(1):4-19, 2000, which is hereby incorporated herein byreference in its entirety. The model expresses the spectral reflectanceof a print as the sum of the reflectance of each combination of ink(called Neugebauer primaries) weighted by the relative proportion of thepaper it occupies. For example, for spot ink S, Cyan, and Magenta, wehave:R(λ)=∝_(o) R _(o)(λ)+∝_(s) R _(s)(λ)+∝_(c) R _(c)(λ)+∝_(M) R _(M)(λ)+∝R_(SC)(λ)+∝R _(SM)(λ)+∝R _(CM)(λ)+∝_(SCM) R _(SCM)(λ)  (2)Where R_(o)(λ), R_(c)(λ), R_(SC)(λ) is a reflectance of substrate, 100%Cyan ink, and overprint of 100% spot and Cyan all printed on substrateat wavelength λ, respectively. Other overprints, such as R_(SCM), aresimilarly defined. Weights ∝ satisfy Demichel equation

$\begin{matrix}{\text{∝}_{o} = {{{\left( {{1 -} \propto_{s}} \right)\left( {{1 -} \propto_{c}} \right)\left( {{1 -} \propto_{M}} \right)} \propto_{M}} = {{{\left( {{1 -} \propto_{s}} \right)\left( {{1 -} \propto_{c}} \right)} \propto_{M} \propto_{CM}} = {\left( {{1 -} \propto_{S}} \right) \propto_{C} \propto_{M} \propto_{S}{= {{\propto_{S}{\left( {{1 -} \propto_{C}} \right)\left( {{1 -} \propto_{M}} \right)} \propto_{SC}} = {{\propto_{S} \propto_{C}\left( {{1 -} \propto_{M}} \right) \propto_{SCM}} = {{\propto_{S} \propto_{C} \propto \propto_{c}} = {{\left( {{1 -} \propto_{S}} \right) \propto_{C}\left( {{1 -} \propto_{M}} \right) \propto_{SM}} = {\propto_{S}\left( {{1 -} \propto_{c}} \right) \propto_{M}}}}}}}}}}} & (3)\end{matrix}$where ∝_(S), ∝_(c), ∝_(M) is spot, Cyan, Magenta ink percentage,respectively.

In order to use the Spectral Neugebauer model with k inks in practice,there is typically a reflectance of 2^(k) Neugebauer primary colorsincluding the color of the substrate, 100% of each ink on its own on thesubstrate, and all 100% ink overprint combinations printed on substrate.Reflectance of substrate, and any overprint of process inks can bederived (or at least approximated) from CIE XYZ values obtained from ICCprofile. Reflectance of 100% of the spot color can be measured or takenfrom an external source such as PANTONE Live (www.pantone.com/live).Reflectance of multiple spot color overprint or process and spot inkoverprint may be either measured from a printed test patch or, fortransparent inks, approximated using product of reflectances. Forexample, reflectance of Cyan and spot color overprint can beapproximated by:

$\begin{matrix}{{R_{SC}(\lambda)} = {{R_{0}(\lambda)} = {\frac{R_{S}(\lambda)}{R_{0}(\lambda)}{\frac{R_{C}(\lambda)}{R_{0}(\lambda)}.}}}} & (4)\end{matrix}$Reflectance of process inks overprint can either be derived from an ICCprofile CIE XYZ value or approximated as a product of individualreflectances normalized for substrate reflectance based on the formulaabove. When inks are approximated by Eq. (4), we obtain:

$\begin{matrix}{{R(\lambda)} = {{R_{0}(\lambda)}{\prod\limits_{i = 1}^{k}{\left( {1 - {\left( {1 - \frac{R_{i}(\lambda)}{R_{0}(\lambda)}} \right)\alpha_{i}}} \right).}}}} & (5)\end{matrix}$Coefficients ∝_(i) in Spectral Neugebauer model are linear inkpercentages before any dot gain correct ion. Demichel equation (3),linear ramp in ∝_(i) results in a linear change of reflectance and thuslinear change of CIE XYZ. To correct for any single-ink non-linearitycaused by the press (often called dot gain), we substitute ∝_(i) in theabove model with gain corrected values g_(i) ⁻¹({circumflex over(∝)}_(i)). Function g_(i) ⁻¹ inverts the dot—gain effect such thatlinear ramp in {circumflex over (∝)}_(i). leads back to linear increaseof reflectance. Several patches of single screened ink can be used toestimate g_(i) ⁻¹ for i-th ink.

Further combinations, aspects, features and description will become evenmore apparent with reference to the following detailed description andaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1a is a diagram showing spectral reflectance of various PANTONEprocess inks as measured using an X-Rite i1Pro spectrophotometer. Italso shows spectrum emitted by red LED at or around 660 nm.

FIG. 1b is a diagram showing a 1931 CIE 2° standard observer matchingfunctions used for converting spectral reflectance to CIE XYZ colorspace.

FIG. 2a is a diagram showing a contrast sensitivity function of humaneye (luminance, red-green, and glue-yellow), with plots of inverse ofmagnitude for luminance, red-green, and blue-yellow sine wave to becomejust noticeable to a human eye as a function of frequency.

FIG. 2b shows printing at different dots per inch (DPI).

FIG. 3 shows spectral sensitivity of a Red LED scanner, with a peak ator around 660 nm.

FIG. 4 shows various colors and their grayscale representation (“scannersees”) as seen by a POS scanner with a 660 nm red LED illumination.

FIG. 5a is an un-watermarked patch of spot color (PANTONE 221 C).

FIG. 5b is a watermarked version of the FIG. 5a patch, with modulatingthe spot color itself.

FIG. 5c is a watermarked version of the FIG. 5a patch, using CMYoverprinting+a screened version of the FIG. 5a patch.

FIG. 5d shows min/max tweaks used to carry the watermark signal in FIG.5 b.

FIG. 5e shows min/max CMY overprint tweaks used to carry the watermarksignal in FIG. 5 c.

FIG. 6a shows a grayscale image as seen by a red LED scanner for theFIG. 5b patch; FIG. 6b shows a grayscale image as seen by a red LEDscanner for the FIG. 5c patch.

FIG. 7 is a diagram showing a spot color data hiding process using CMYoverprints.

FIG. 8a shows a grayscale representation of a watermark tile printed at75 watermark pixels per inch; FIG. 8b shows a corresponding signal valuehistogram of the FIG. 8a tile.

FIG. 9 illustrates two (left and right) mid gray patches, with the leftpatch embedded using CMY tweaks, and the right patch using only Cyantweaks.

FIG. 10 illustrates two tints that can be used for marking white, textbased or open spaces. The left tint includes a cyan, magenta and yellowinks whereas the right patch only includes cyan.

FIG. 11 is a diagram showing optimization tradeoffs when selecting ascreened spot color and corresponding process colors.

FIG. 12 depicts a product package including multiple different areas,the different areas including different data hiding therein.

FIG. 13 is a flow diagram for a color embedding process.

FIG. 14 show a watermarked gray patch (top left), watermark tweaks in aCyan plane (top right, with a value of 11.1), watermark tweaks in aMagenta plane (bottom, left, value of −9.6), and watermark tweaks in aYellow plane (bottom right, value of 6.1),

FIG. 15 is a diagram showing relative signal standard deviation forsignal capture with a monochrome sensor with red illumination & red andblue illumination and white illumination (with an RGB sensor)

FIG. 16 shows relative illumination timing.

FIG. 17 is a plot showing various colors and their reflectancepercentages.

FIG. 18 is a color palette corresponding to the plot in FIG. 17.

FIG. 19 is a graph showing percent reflectance at different wavelengthsfor various colors.

FIG. 20a is a diagram showing warping for a design when packaged.

FIG. 20b is a diagram showing distortion due to packaging correspondingto the designs in FIG. 20 a.

FIG. 21 is diagram showing scanning of a packaged product at differentscanning speeds.

FIG. 22 is a bar chart showing percentage of successful watermarkdetections at different watermark resolutions (50 & 70 watermark perinch) and increasing scanning speeds/inch in a simulated test.

FIG. 23 is a flow diagram for a package design and data hiding workflow.

FIG. 24 shows an example user interface for a plugin to achieve spotcolor embedding.

FIG. 25 is a block diagram for a color visibility model.

FIG. 26 is a block diagram for a color visibility model with CSFs variedaccording to image local luminance.

FIG. 27a is a block diagram for a color visibility model to achieveequal visibility embedding (EVE).

FIG. 27b is a diagram showing EVE embedding compared to uniformembedding.

FIG. 28a corresponds to Appendix B's FIG. 1, which shows a quality rulerincreasing in degradation from B (slight) to F (strong).

FIG. 28b corresponds to Appendix B's FIG. 2, which shows thumbnails ofthe color patch samples with a watermark applied.

FIG. 29 corresponds to Appendix B's FIG. 3, which shows a mean observerresponses with 95% confidence intervals for color patches.

FIG. 30 corresponds to Appendix B's FIG. 4, which shows mean observerresponse compared with a proposed visibility model.

FIG. 31 corresponds to Appendix B's FIG. 5, which shows mean observerresponse compared with the proposed visibility model with luminanceadjustment.

FIG. 32 corresponds to Appendix B's FIG. 6, which shows mean observerresponse compared with S-CIELAB.

FIG. 33 corresponds to Appendix B's FIG. 7, which shows watermarkembedding with uniform signal strength (left) and equal visibility froma visibility model (right). The insets are magnified to show imagedetail.

FIG. 34 corresponds to Appendix B's FIG. 8, which shows visibility mapfrom uniform signal strength embedding (left) and equal visibilityembedding (right) from FIG. 18.

FIG. 35 corresponds with Appendix B's FIG. 9, which shows Apple tart,Giraffe stack and Pizza puff design used in tests.

FIG. 36 is a flow diagram for obtaining one or more substitute spotcolors.

FIG. 37 is a flow diagram for a related process for determining one ormore substitute spot colors.

Other drawings are included throughout the text in Appendix A, Reed etal., “Watermarking Spot Colors in Packaging,” which is herebyincorporated herein by reference.

DETAILED DESCRIPTION

There are four (4) main sections that follow in this DetailedDescription (I. Adaptive Embedding Framework; II. Spot Color and ProcessColor Data Hiding; III. Additional Implementations and Description; andIV. Implementations of Adaptive Embedding Framework). These sections andtheir assigned headings are provided merely help organize the DetailedDescription. Of course, description and implementations under one suchsection is intended to be combined and implemented with the descriptionand implementations from the other such section. Thus, the sections andheadings in this document should not be interpreted as limiting thescope of the description.

I. Adaptive Embedding Framework

Portions of this disclosure are described in terms of, e.g., data hidingfor product packaging (sometimes just referred to herein as “packaging”or “package”) and other printed objects. These techniques can be used,e.g., to alter or transform how color inks are printed on variousphysical substrates. The alterations or transformations preferablyresult in a printed design carrying machine readable indicia. Such datahiding techniques may beneficially interrelate with the adaptiveembedding framework below.

1. Design of Human Visual System (HVS) Models:

A human visual system model is used to indicate the extent to whichchanges to an image will be visible. While a watermark signal may bedesigned so that it is less noticeable by constructing the signal withless noticeable colors or spatial structure, the more sophisticatedmodel analyzes the change in visibility relative to the host signal.Thus, a watermark embedding process should consider the extent to whichthe changes made to an existing image are visible. The host image mayhave little or no variation, or even no color content, in which case thevisibility model assesses visibility of the watermark signal itself andproduces output providing a measure of visibility. A watermark embedderfunction adapts the watermark signal amplitude, color and spatialstructure to achieve a visibility target which depends on theapplication. For example, a fashion magazine would have a lowervisibility target than packaged goods. The host image may have regionsof color tones, in which case, the embedder considers color errorsintroduced by the embedding process in those regions. In many cases, ahost image includes regions with different color and spatial attributes,some uniform, others variable. In areas of the host image withvariability, the changes due to embedding should be adapted to take intoaccount not only visibility of a watermark signal, but in particular,visibility relative to the host signal, and its masking of changes dueto the watermark embedding.

a. Watermark Signal Design:

The watermark signal is designed to be minimally visible within thetypes of host image content in which it will be embedded. This designincludes selecting attributes like spatial frequency content andpseudorandom spatial patterns that tend to be less visible. Someexamples of such implementations are described in U.S. Pat. No.6,614,914, which is hereby incorporated by reference in its entirety.The watermark signal need not have random properties, however. It mayhave a regular or repeated pattern structure that facilitates robustdetection and reliable data extraction as detailed in our application62/106,685, entitled Differential Modulation for Robust Signaling andSynchronization, which is hereby incorporated by reference in itsentirety. The watermark design also preferably leverages encoding incolor channels to optimize embedding for visibility and robustness asdescribed in US Published Application 20100150434, which is alsoincorporated by reference in its entirety.

b. Human Visual System (HVS) Models for Watermarking:

Prior work in HVS modeling provides at least a starting point fordesigning HVS models for watermarking systems. See, in particular, ScottJ. Daly, “Visible differences predictor: an algorithm for the assessmentof image fidelity”, Proc. SPIE 1666, Human Vision, Visual Processing,and Digital Display III, 2 (Aug. 27, 1992); doi:10.1117/12.135952, andU.S. Pat. No. 5,394,483 to Daly, entitled, Method and apparatus fordetermining visually perceptible differences between images, which arehereby incorporated by reference in their entirety. Daly's HVS modeladdresses three visual sensitivity variations, namely, as a function oflight level, spatial frequency, and signal content. The HVS model hasthree main components: an amplitude non-linearity function in whichvisual sensitivity is adapted as a non-linear function of luminance, aContrast Sensitivity Function (CSF) model of the eye that describesvariations in visual sensitivity as a function of spatial frequency, anda model of masking effects. The first component is an amplitudenon-linearity implemented as a point process. The CSF can be implementedas a filtering process. The third in the sequence of operations is adetection process. The output is a map of the probability of detectingvisible differences as a function of pixel location.

Daly used the HVS in U.S. Pat. No. 5,394,483 to develop a method ofhiding one image in another image. See, U.S. Pat. No. 5,905,819 to Daly,Method and apparatus for hiding one image or pattern within another,which is hereby incorporated by reference in its entirety. Another HVSis described in U.S. Pat. No. 7,783,130 to Watson (also published as USApplication Publication 20060165311), entitled Spatial StandardObserver, which is hereby incorporated by reference in its entirety.

In our prior work, we developed a perceptual masking model forwatermarking that incorporates a CSF of the eye as well as a method fordirectional edge analysis to control perceptibility of changes due towatermark embedding around directional edges in a host signal. See U.S.Pat. No. 6,631,198, which is hereby incorporated by reference in itsentirety.

We found that the Daly and Watson methods were useful but further workwas needed for our watermarking techniques in color channels. Therefore,we developed HVS methods that incorporate color visibility models.

Our application Ser. No. 13/975,919 (U.S. Pat. No. 9,449,357) describesa full color visibility model for watermarking in color channels. U.S.application Ser. No. 13/975,919, entitled Geometric Enumerated WatermarkEmbedding for Spot Colors, is hereby incorporated by reference in itsentirety. One particular usage is watermarking in color channelscorresponding to color inks used to print a host image. The watermarkmodulations of color values are modeled in terms of CIE Lab values,where Lab is a uniform perceptual color space where a unit difference inany color direction corresponds to an equal perceptual difference. TheLab axes are scaled for the spatial frequency of the watermark beingencoded into the image, in a similar manner to the Spatial CieLab model.See, X. Zhang and B. A. Wandell, e.g., “A spatial extension of CIELABfor digital color image reproduction,” in Proceedings of the Society ofInformation Display Symposium (SID '96), vol. 27, pp. 731-734, San Jose,Calif., USA, June 1996, which is hereby incorporated by reference in itsentirety.

This scaling provides a uniform perceptual color space, where a unitdifference in any color direction corresponds to an equal perceptualdifference due to the change made to encode a watermark signal at thatspatial frequency. The allowable visibility magnitude is scaled byspatial masking of the cover image. This masking is computed based on amasking function. Examples of masking functions include the maskingcomponents of the Spatial Standard Observer model of Watson or the HVSmodels of Daly referenced above, as well as our prior patents, such asU.S. Pat. Nos. 6,631,198 and 6,614,914, referenced above.

Relatedly, our application Ser. No. 14/588,636 (U.S. Pat. No.9,401,001), describes techniques for embedding watermarks in colorchannels that employ full color visibility models. Patent applicationSer. No. 14/588,636, entitled Full-Color Visibility Model Using CSFWhich Varies Spatially with Local Luminance, is hereby incorporated byreference in its entirety. This approach uses a full color visibilitymodel for watermarking in color channels. This visibility model usesseparate CSFs for contrast variations in luminance and chrominance(red-green and blue-yellow) channels. The width of the CSF in eachchannel can be varied spatially depending on the luminance of the localimage content. The CSF is adjusted so that more blurring occurs as theluminance of the local region decreases. The difference between thecontrast of the blurred original and marked image is measured using acolor difference metric.

The luminance content of the host image provides potential masking ofchanges due to watermarking in chrominance as well as luminance.Likewise, the chrominance content of the host image provides potentialmasking of changes due to watermarking in chrominance as well asluminance. In our watermarking systems that embed by changes inluminance and chrominance, or just chrominance, of the host image, theembedding function exploits the masking potential of luminance andchrominance content of the host image. The masking potential at a givenregion in an image depends in part on the extent to which the host imageincludes content at that region that masks the watermark change. Forexample, where the watermark signal comprises mostly high frequencycomponents, the masking potential of the host image is greater atregions with high frequency content. We observe that most high frequencycontent in a host image is in the luminance channel. Thus, the luminancecontent of the host is the dominant contributor to masking potential forluminance changes and chrominance changes for high frequency componentsof the watermark signal.

In some applications, the watermark signal has lower spatial frequencycontent, and the embedding function computes the masking capability ofthat low frequency content on the watermark signal as well, taking intoaccount both luminance and chrominance masking on luminance andchrominance components of the watermark signal.

Our watermarking techniques in luminance and chrominance channels alsoleverage masking of spatial structure particular to those channels. Suchvisibility effects originate both from the host image as well as theprint technology. The host image content can have strong spatialfrequencies at an angle, which masks similar spatial structure of thewatermark at that angle. Likewise directional edges in the host imagecontrol watermarking along the edge as noted in U.S. Pat. No. 6,631,198.

The print technology sometimes prints with halftone screen or raster fordifferent inks with different orientation, shape, and structure. Blackinks, for example, are sometimes printed with halftone dots at screenangle of 45 degrees to achieve a higher print quality because black ismost noticeable to the eye and it is desirable to make the spatialpattern of black dots less noticeable. These types of print structuresfor different color inks provide an opportunity to hide the watermarksignal differently in the color channel or channels that correspond tothat ink. For more on watermarking that exploits the halftone structureand Raster Image Processor used in printing, please see our US PatentPublication 2014-0119593, which is hereby incorporated by reference inits entirety.

2. Robustness Modeling:

Optimizing the embedding for robustness adds another constraint in whichthe encoding is controlled not only to achieve a desired visual quality,but also to achieve reliability in decoding the watermark. A simple viewof robustness may be to set a floor on the gain or signal level of thewatermark signal, but this is potentially less useful if it does notconsider how well watermark signal structure is maintained within a hostimage, or opportunities to apply less gain where signal structure ismaintained due to attributes of the host image that are inherentlybetter at carrying data with less or no modification. A moresophisticated view is to consider how the watermark signal conveys datathrough its color and structure or the color and structure created whenit exploits host signal structure to mask watermark variations and/orcarry data (e.g., where signal data is encoded in the relationship amongvalues or an attribute derived from a region in an image, how is thatrelationship or attribute impacted by modifications made to reducevisibility?) Thus, controlling the strength of the watermark signalshould also ensure that such control does not undermine its reliability.A robustness metric can be designed based on readability of thewatermark, e.g., through a detection metric: modification of the signalto remain within a visibility constraint should maintain the structureof the signal that conveys digital data. Our application Ser. No.13/975,919 describes a framework for watermark embedding that optimizesembedding based on visibility and robustness models. See Appendix A ofSer. No. 13/975,919: Bradley, Reed, Stach, “Chrominance watermark embedusing a full color visibility model.”

3. Modeling the Distortion of the Channel:

Related to robustness optimization, the embedding process should takeinto account the impact of anticipated distortion introduced byprinting, use or scanning of the printed object. A particular concern isthe extent to which a change to embed an image will become more visibledue to the technology used to render the image, such as the display orprinter. This type of rendering distortion may be incorporated into themodel to predict the change in visibility and/or robustness afterdistortion, and adjust the embedding to compensate for this change.Likewise, the rendering distortion may also impact robustness. As such,robustness modeling should account for it as well.

See in particular, our U.S. Pat. No. 7,352,878, which describes a modelthat incorporates a model of the rendering device (e.g., display orprinter) within an adaptive embedding function. The embedder uses thismodel to adapt the visibility mask used to control the watermark signal,so that it takes into account the effects of the rendering device onvisibility. U.S. Pat. No. 7,352,878 is hereby incorporated by referencein its entirety. These techniques may be further combined with fullcolor visibility models and robustness models referenced in thisdocument.

Other examples of modeling distortion include adding noise, applying ageometric distortion, compressing the image, and modeling image capturedistortion. For package images to be printed on a 3D object with knownshape, the geometric distortion applied to the image is known and itseffect can be compensated for in the embedding of the watermark in thepackage design. Examples include labels wrapped around a curved object(e.g., a yogurt cup or soup can). The watermark signal (and in somecases the host signal itself) may be pre-distorted to compensate for thegeometric transformation caused by application of it to the object. Thisand other noise sources may be modeled and applied to the watermarkedimage to measure its reliability in the robustness model. Thewatermarking process is then corrected or iterated as necessary toachieve reliable detection metrics.

4. Printing Technology Limitations:

Another related constraint is the limitation of the print technology. Asnoted, it may cause distortion that impacts visibility and robustness.It may have limitations in the manner in which it is able to represent acolor or spatial structure of the watermark signal. It may not be ableto print a particular color, dot structure, orientation orsize/resolution, or may introduce registration errors among differentink layers that make encoding in color directions not viable. Distortiondue to dot gain and other limitations of replicating an image on asubstrate need to be accounted for. Dot gain distortion can be modeledin the robustness model such that the watermark signal is embedded to berobust to the distortion.

5. Image Capture Device Limitations:

Another design consideration is the image capture device. Some forms ofimage capture devices, such as barcode scanners, do not capture fullcolor images. For example, some barcode scanners have monochrome imagesensors and illuminate an object with red LED illumination. This type oflimitation requires that the watermark signal be designed so that it canbe “seen” by the capture device, meaning that at least a portion of thewatermark signal is readable in the spectral band or bands captured bythe image sensor. We discuss these limitations and methods foraddressing them in our US Application Publication 2013-0329006 and U.S.Provisional Application 62/102,247, which are hereby incorporated byreference in their entirety.

6. Color Appearance and Attention Models:

Attention (also referred to as “saliency”) models may also be includedto adjust visibility model for controlling watermark modification at aparticular location within an image. See our U.S. patent applicationSer. No. 14/588,636 for description of how to use this type of model ina watermark embedder. An attention model generally predicts where thehuman eye is drawn to when viewing an image. For example, the eye mayseek out flesh tone colors and sharp contrast areas. One exampleattention model is described in Itti et al., “A Model of Saliency-BasedVisual Attention for Rapid Scene Analysis,” IEEE TRANSACTIONS ON PATTERNANALYSIS AND MACHINE INTELLIGENCE, VOL. 20, NO. 11, November 1998, pgs.1254-1259, which is hereby incorporated herein by reference in itsentirety. High visual traffic areas identified by the attention model,which would otherwise be embedded with a relatively strong or equalwatermark signal, can be avoided or minimized by a digital watermarkembedder, e.g., through adjustment of the visibility map used to controlapplication of the watermark signal to a host image. In many applicationscenarios, it is advantageous for the embedding system to take intoaccount a Color Appearance Model (CAM) to assess the extent to which achange in color is likely to be noticeable relative to colors present inthe host image. For information on CAM, please see Fairchild, Mark D.Color Appearance Models. Chichester: John Wiley & Sons, 2013. Ourapplication of digital watermarking in packaging provides methods inwhich CAM is automated and applied in embedding functions foradvantageous effect.

Package designs typically include colors for which the package designerattaches importance over other colors. For example, a consumer productbrand may have a color or combination of colors that are stronglyassociated with the brand. The designer, thus, seeks to achieveconsistency and accuracy in representing this color across all of itspackages. This may be achieved through the use of a spot color. Anotherexample is where the designer selects a particular color or colorcombination to evoke a particular theme for the product (e.g., apineapple flavored product might use a yellow color). This color mightbe modified by the watermark, but the modification should not underminethe intent of the designer, nor appear objectionable to the consumer.Finally, the remaining colors on package may be less important, andthus, more available for modification. Among these parts of the packagedesign, there may be regions in which a tint may be applied to conveythe digital watermark or the host image may be modulated in a particularcolor or set of colors. Overall, none of the image should be modified ina manner that undermines the designer's objective for the dominant brandcolors, or an important thematic color.

To illustrate, consider an implementation of adaptive watermarkembedding in a plug in of a design program used for designing thepackage image. The plug in allows the designer to specify importance ofcolors, which in turn, dictates whether the plug in will modify a color,and if so, the extent to which the modification are allowed to deviatefrom the original color. For the colors in a design, the CAM takes theirpriority and provides constraints for color modifications that areapplied in the embedding function. The color match error for use of asubstitute color for an original color (e.g., process inks for spotcolor) and the color error introduced by the watermark are weightedaccording to the priority of the color. Additionally, the CAM places aconstraint on the direction in color space of the modification to aparticular color. The following examples will illustrate.

If a bright background area is available for conveying a data signal,the CAM detects the bright area by their pixel values and providesspecification for the tint used to fill that area that satisfies the CAMconstraint relative to the color of other features in the design. Thisbright background is intended to look white or nearly white and a lighttint added to it will not be noticeable so long as it is uniform in thedesign and not modulated in a color direction that is incompatible withthe color of neighboring features. So long as the area covered by thetint remains substantially brighter than the rest of the designelements, it will not be noticeable. It would only be noticeable if itwere positioned next to a blank area with no tint. The CAM constraintspreclude noticeable changes of appearance of regions and can also be setso that the modulation of such areas are smoothly tapered near regionswith other colors of higher importance.

Another example is a package design where there is a thematic color forwhich the CAM limits the direction of color modulation or alternativelyspecifies a black tint to convey the watermark signal. The example ofthe yellow for a pineapple product is appropriate to illustrate. Forsuch a case, the CAM takes the priority weighting for the yellow andfurther constrains the modulation color direction to precludeobjectionable color changes within the yellow region of the package.Green is an example of a color that would be incompatible and thusprecluded by the constraint set by the CAM for the yellow region of thedesign. Alternatively, the embedder substitutes a black ink tint if arobustness measure indicates that a reliable signal cannot be achievedin allowable chrominance modulation directions or channels.

II. Spot Color and Process Color Data Hiding

A significant number of packages in commerce are printed to include atleast some areas using “spot colors” as discussed above. Spot colors mayinclude, e.g., custom pre-mixed ink designed to achieve a certain colorwhen printed on a specified substrate. PANTONE is one example of a spotcolor system that is commonly used in the product packaging industry.Packages may also include so-called process colors. As discussed above,process colors typically refers to Cyan (C), Magenta (M), Yellow (Y)and/or Black (K) inks that are used to simulate a wide range of colorsby mixing these various inks on a substrate. Process colors can beprinted with, e.g., half tone dots.

Data hiding within a spot color can be challenging since the spot colorcan be viewed as a flat patch, with little or no variance. Modulating aflat color patch to carry an information signal may introduce colorshifts and noticeable visible artifacts. Also, many package designersuse spot colors to achieve a distinctive color. Altering a specific spotcolor may result in aesthetic complaints from the designers anddeviation for the distinctive color.

This disclosure provides, e.g., methods, systems, software plugins andapplications, and apparatus for hiding information in spot colors andother color areas while minimizing color shifts and visibility concerns.In some cases we prefer to hide data in spot colors in a chrominancedomain rather than with luminance to reduce the visibility of the hiddendata.

With reference to FIG. 12, a product package 10 may include multipledifferent printed areas 12, 14, 16. Area 12 may include spot color ink,area 14 may include one or more process color inks and area 16 mayinclude printed text. Package 10 may include multiple different forms ofdata hiding to convey machine-readable indicia over a substantialportion of the package. For example, the area 12 spot color may bescreened and then overprinted with CMY process colors to covey awatermark signal, the area 14 may already include process colors, whichcan be modulated to convey an information signal. For area 16 (and anywhite spaces) a subtle tint of CMY(K) may be printed which includes aninformation signal. The information signals in theses area preferablyinclude overlapping (or the same) information such that the informationsignal can be obtained from signal detection in one or more of the areas12, 14, 16. In fact, for many package designs we prefer to redundantlyembed an information signal over substantially all package surfaces(e.g., 80-100% coverage).

One form of an information signal used to guide embedding can be arobust spread spectrum digital watermark signal. One instance may carrya plural-bit payload, e.g., a 47-bit payload, enough to encode the sameinformation as is carried in a Global Trade Item Number (GTIN-14) oftenfound in a linear UPC barcode. A watermark payload may also includeadditional error correction bits, checksums, payload version bits andother information. A watermark carrying a specific payload can berepresented, e.g., at a spatial resolution of 75 DPI, as a 128×128 pixelgrayscale image, called a watermark tile. FIG. 8a shows one instance ofa watermark tile with a histogram (FIG. 8b ) of watermark signal values.Of course, different instances of watermarking and other types ofmachine-readable indicia can be used instead of the illustratedwatermark tile. In the illustrated case, the watermark tile includes azero-mean signal with positive and negative values referred to as“waxels” (e.g., watermark pixels). Watermark tiles can be concatenatednext to each other to cover larger area. As opposed to a traditional UPCbarcode, portions of a single watermark tile can be cropped and stillsuccessfully be decoded due to its repetitive structure. When printresolution is different from 75 DPI (e.g., see FIG. 2, shaded area), thewatermark tile can be up-sampled to match the print resolution.

Point of Sale (POS) scanners with red LEDs typically include anarrow-band monochromatic imager with a peak response at or around 660nm. Such red LED scanners are often found at grocery checkout lanes,looking for traditional UPC barcodes. See FIG. 3 for a spectral responseof a typical red LED capture device; see also FIG. 1a . A red LEDcapture device (e.g., a point of sale scanner or camera) only “sees”colors which reflect at or around 660 nm. If a color strongly reflectsat this wavelength the captured device ‘sees’ white. Bright yellows,magenta, pink, orange and white are all ‘seen’ as white by a red LEDcapture device. If a color has a low reflection at this wavelength(e.g., absorbs the wavelength) the captured device “sees” black. Darkblue, Cyan, green, purple and black are all ‘seen’ as black by thecamera. FIG. 4 illustrates these arrangements. Thus, changes made tocolors of low spectral reflectance at or around 660 nm are visible(e.g., register as black pixel values) to the scanner and thus aresuitable for carrying information signals. Due to the combination of anarrow-band illumination and monochromatic sensors, a typical barcodescanner can only see grayscale images created by spatial changes in inkreflectance at 660 nm. If more inks are overprinted, the grayscale valueG can be obtained from a 660 nm component of the Spectral Neugebauermodel as:G=sensitivity·R(660 nm)+offset  (6)

Two approaches are now considered when introducing an information signalinto a spot color. With reference to FIG. 5a , FIG. 5b and FIG. 5d , afirst approach modulates the spot color itself (FIG. 5a ). Max (FIG. 5d, left patch) and Min (FIG. 5d , right patch) tweaks (e.g., pixel orcolor changes, color value amounts, and/or signal or color channelmodulations or modulation changes) are determined to carry a signal. TheMax patch can be a 100% version of the spot color, and the Min patch canbe a screened back version of the spot color (e.g., 85% screen). Themin/max tweaks are substituted for (or interpolated according to aninformation signal) the original spot color values across the spot colorpatch according to an information signal, e.g., the original signal ismodulated with the min/max tweaks to convey the information signal.

A second approach uses CMY min and max tweaks (See FIG. 5e ) as a tintapplied (e.g., printed) over a screened back version of the spot colorin FIG. 5a . The resulting watermarked patch (see FIG. 5c ) is a closerapproximation to FIG. 5a relative to FIG. 5b . Thus, an informationsignal can be conveyed through a CMY tint in a combined screened spotcolor+process colors. The combined screened spot color+process colorsare provided to approximate the original spot color. CMY tweaks orsignal modulations can be added with a tint over (or, in some cases,beneath) a spot color screen. As discussed above, the term “screen”implies that a spot color is scaled back or reduced, e.g., in terms ofits color percentage or chrominance values. Watermark tweaks (ΔC, ΔM,ΔY) can be provided to achieve a scanner signal, ΔGRAY, as seen by amonochrome scanner. One optimization combines a CMY tint including min(e.g., negative) and max (e.g., positive) watermark signal tweaks thatcombine to approximate an original spot color.

FIG. 6a and FIG. 6b presents a red LED capture device grayscale viewpoint of the FIG. 5b and FIG. 5c embedded patches. The detectablesignals from each patch are very similar in terms of standard deviation,2.2 (FIG. 6a ) and 2.1 (FIG. 6b ). Yet the improvement in visibilityreduction in FIG. 5c vs FIG. 5b is tremendous. This points to anadvantage of the combined screened spot color+process color tint.

This second approach is described even further below with respect toFIG. 7. See also Appendix A, Reed et al., “Watermarking Spot Colors inPackaging,” which is hereby incorporated herein by reference in itsentirety, for a related disclosure.

With reference to FIG. 7, an embedding process is shown relative to anun-watermarked spot color, PANTONE 221 C. The below numbered 1-6paragraphs correspond to numbers 1-6 in FIG. 7.

1. Determine CMY values for overprinting with a screened spot color. Thecombined screened spot color+process colors (CMY) are provided toapproximate the original spot color. We generally use the term “tint” torefer to the selected CMY process colors. The CMY approximation or tintcan be determined by testing, or models of overprinting can also be usedas discussed, e.g., in Deshpande, K. and Green, P. “A simplified methodof predicting the colorimetry of spot colour overprints,” 18th ColorImaging Conference: Color Science and Engineering Systems, Technologiesand Applications, pg. 213-216, San Antonio, USA 2010, which is herebyincorporated herein by reference in its entirety. Or, for a givensubstrate, a PANTONE ink swatch can be scanned with a spectrophotometerto determine corresponding process color (or L*a*b*) correspondence.Libraries, table and/or indices of such approximations, predictions ormeasurements can be built for rapid consultation. Interpolation can beemployed to estimate process color tint percentages for values notexplicitly represented in the table. For example, a table or library canbe accessed to find CMY values, which when combined with a screened backversion of a particular spot color, will yield a close approximation tothe original spot color.2. Screen spot color. Screening provides information signal headroom foran over (or under) printed CMY tint. Recall from above that the CMY tintwill carry the information signal. An amount of screening may depend,e.g., at least in part on an amount of cyan absorption associated withthe original spot color. In the illustrated PANTONE 221 example, thespot color is screened to 75%. Of course, this percentage screen is notlimiting as other percentages may be chosen based on, e.g., visibility,robustness and masking considerations. For this example, it wasdetermined that a color approximation of the illustrated PANTONE 221spot color in terms of percent (%) spot, C, M and Y: 75%, 13, 57, 8.3. Simulate the CMY overprint+screened spot color to evaluate colormatch error E_(CM) between the original spot color and the CMYoverprint+screened spot color. This process can be used to iterateselection of the CMY tint values to minimize error of the selectedprocess colors and screen. For example, ΔE₇₆, ΔE₉₄, ΔE₂₀₀₀, metrics canbe used to minimize color error between the process color tint+screenedspot color and the original 100% spot color. Different screenpercentages and CMY tint values at or around the predicted colors can beinvestigated to find values with minimized error.4. Decompose CMY tint into Min and Max Tweaks. For example, a gradientsearch process or least squares distance process can be conducted tofind optimum tweaks.

These processes may consider other factors as well. For example, anoptimization process may consider the original spot color, visibilityconstraints, robustness requirements at a particular spectral response(e.g., at 660 nm), a k (black) channel, other spot colors, etc. For theillustrated example, Min Tweak (%): 75, 27, 42, 16; and Max Tweak (%):75, 0, 73, 0 were determined. In FIG. 7, E_(WM)=watermark error, whichcan be a weighted sum of ΔL*, Δa* and Δb* where CIE ΔL* can be weightedheaviest to account for greater visibility of the watermark signal inlightness, followed by Δa* (red-green) and then Δb* (yellow-blue). Somegoals of the Min and Max CMY Tweak may include, e.g., i) similarperceived visibility to original spot color when overprinted withscreened spot, ii) maximum tweak difference at 660 nm (corresponding tored LED scanner), which maximizes scanner visibility; iii) minimumluminance (CIE L*) difference, recall that human visual system is mostsensitive to luminance changes and color difference (CIE a* and b*)should be kept under control.

5. Embed an information signal by spatially and variously changing someor all of the CMY tint between Min and Max tweaks. The tweaks can beused to modulate or transform the CMY tint to convey an informationsignal. An information signal (e.g., as carried by a watermark title inFIG. 7) can be embedded into CMY tint by interpolating between min andmax tweaks based on watermark tile values. For example, the followingequation can be used to interpolate tweak values for a given pixel orarea value: T_(CMY)=T_(Min)+(1+W)(T_(Max)−T_(Min))/2, where W is agrayscale value of the watermark tile at a certain location and Tmin andTmax are the min CMY tweak and max CMY tweak values, respectively. Theresult of this process for each watermark title value yields a modulated(e.g., watermarked or embedded) CMY tint. Of course, other types ofencoding besides digital watermarking can be used to guide embedding ofthe CMY tints. For example, 2D barcodes, UPC barcodes, and other typesof machine readable indicia, can be used to guide construction of aninformation signal to be hidden or embedded via CMY tint.6. The modulated CMY tint is overprinted on the screened back spot colorto yield a marked or embedded spot color.

Additional color blending, signal considerations, embedding and tweakvalue determination details, etc., are discussed below.

When embedding an information signal like a watermark tile into artwork,visibility of the embedded information signal as observed by a humanuser is often balanced with watermark robustness when scanning apackage. Based a scanner response under red LED illumination shown inFIGS. 3 and 4, ink changes printed in Cyan or Black are visible to thescanner and thus can be useful for carrying an information signal. Thehuman visual system, however, is significantly more sensitive toluminance changes (e.g., caused by changes in black ink) thanchrominance changes (e.g., caused by changes in cyan ink). To reducehuman visibility a watermark tile can be embedded by modifying each ofthe C, M, Y channels with grayscale values of the watermark tile Wweighted by elements of the unit-length color weight vector ω=(ω_(C),ω_(M), ω_(Y)) and global signal strength σ:C _(i) ′=C _(i)+σω_(C) W _(i) ,M _(i) ′=M _(i)+σω_(M) W _(i) ,Y _(i) ′=Y_(i)+σω_(Y) W _(i),  (7)where index i denotes the pixel of each color separation. Color weightsω drive the color of watermark signal, while σ changes the overallstrength of the signal. Both parameters influence the visibility of thewatermark.

In general, the color weights ω may be associated with an ICC colorprofile for CMYK artwork which captures the color of CMYK inkoverprints. For a typical GRACoL profile, the color weights can be setto, e.g., ωGRACoL for CMY=(0.69, −0.61, 0.39). Even though a red LEDcapture device does not see magenta and yellow changes, non-zero weightscan be chosen to help minimize luminance changes introduced by embeddingthe watermark signal in Cyan. FIG. 9 shows sample patches embedded withthis method. The left patch in FIG. 9 shows a watermark tile embedded ina mid-gray patch using the color weights mentioned in this paragraph.The right patch in FIG. 9 shows the same watermark tile embedded in themid-gray patch with only changes introduced in a Cyan channel. The Cyanonly patch (right patch) is visibly “grainy” or “noisy” compared to thechrominance (left patch, CMY embedded) patch.

Due to spectral dependency of a red LED capture device, only aninformation signal embedded in Cyan separation is available to thedetector because signal in Yellow and Magenta are not seen by thecapture device. In case of the ωGRACoL color weights mentioned above(0.69, −0.61, 0.39), this only represents about 0.69²=48% of the totalsignal energy embedded in the artwork that is extracted by a captureddevice. If a full-color image sensor is available, such as in a smartphone, the embedded watermark signal present in all CMY plates can becombined by aligning a grayscale conversion weight w:

$\begin{matrix}{G = {{{sensitivity} \cdot {\sum\limits_{\lambda}{{w(\lambda)}{R(\lambda)}}}} + {offset}}} & (8)\end{matrix}$

In RGB color space, this grayscale conversion can be approximated as0.52·R−0.81·G+0.29·B.

When a CMY ink combination is overprinted with Black (K) to producedarker colors, the Black ink may act like an optical filter and reducemagnitude of changes introduced in Cyan separation. This may lead to aweaker watermark signal as seen by a red LED scanner and thus therobustness of the watermark can be degraded. This loss can becompensated either by increased signal strength σ, or by replacing aportion of the Black ink with a CMY combination making the final CMYKmix more suitable for watermarking, e.g., using a process known as UnderColor Addition. Colors with either no or 100% Cyan component poseanother challenge. If Eq. 7 is applied blindly, half of the waxels maynot be embedded due to clipping resulting in a robustness loss. Thiscould be resolved by compressing the color gamut of the artwork. Forexample an image with no Cyan, 2%-4% Cyan ink can be added in theoriginal design. A watermark can then be inserted in thispre-conditioned artwork using methods described above.

In order to utilize the full potential of digital watermarking forproduct packaging, a large portion of the package surface can bewatermarked (e.g., 80-100%). That is, a package may include manyredundant instances of an information signal hidden therein. Somepackages contain significant areas without any ink coverage. Such areasmay lead to dead zones and reduce the full benefit of improved checkoutspeed. To resolve this, white areas can be covered by a light CMY tintwhich is can be modulated prior to printing to carry informationsignals. Tint including 4% C, 2% M and 2% Y can be used for offsetprinting white, open or behind text areas. An example of this tint isshown in FIG. 10, where the left patch includes the suggested 4% C, 2% Mand 2% Y tint, and the left patch includes just cyan only.

When embedding a watermark in complex artwork, not just flat spot colorareas as is the one of the focuses of FIG. 7, more signal can beembedded in textured areas than in flat regions because image texturemasks the presence of the watermark signal. To better utilize thiseffect, both direction of the color weight vector and signal strengthmay vary spatially to equalize visibility of the watermark. This can bedone either manually in image editing software or automatically asdiscussed in Reed et al., “Full-color visibility model using CSF whichvaries spatially with local luminance.” In Q. Lin, J. P. Allebach, andZ. Fan, editors, Proceedings SPIE 9027, Imaging and Multimedia Analyticsin a Web and Mobile World 2014, volume 9027, 2014, which is herebyincorporated herein by reference in its entirety. Similarly, form factorof a package may indicate that the signal strength to be increased. Forexample, the area of a soda can is significantly smaller than that of acereal box. Larger objects provide more opportunities for a capturedevice to successfully read a hidden signal. So the soda can may have asignal strength that is stronger relative to the cereal box. Similarly,the geometry of a package may dictate increased signal strength. Forexample, curved surfaces and near corner areas (e.g., packaging for acan of soup or corner of a package, etc.) may warrant a stronger signalstrength. 3D models or user defined areas can be used to identify cornerareas for increase signal strength, or for curved surfaces, the signalstrength across the entire surface can be increased.

Returning to the spot color embedding discussed relative to FIG. 7, thecolor difference between 100% spot and the screened 75% with overprintedCMY tint, denoted as E_(CM), can be referred to as Color Match Error andcan be measured using various color difference metrics as mentionedabove. The CMY tint can be decomposed into so-called Max and Min Tweaksby, e.g., minimizing weighted color difference between them whileachieving a detectable difference in spectral reflectance at 660 nm whenoverprinted with the screened 75% spot color.

Color difference between min and max tweaks overprinted with 75% spot,denoted E_(WM), is called Watermark Error. A final watermark can beproduced by overprinting 75% screened spot and the modulated CMY tint.In this process, both color errors are interconnected. In order to keepluminance changes minimal more space for CMY tweaks can be used andthus, possibly, increasing the color match error, E_(CM). A spot screenof 75% is also a parameter that could be changed. Difference of spectralreflectance at 660 nm, denoted as Δ₆₆₀ serves as a measure of watermarksignal strength similar to parameter σ in Eq. (7). Given a value of Δ₆₆₀spectral ink overprint models can be used to find optimal value of spotscreen and min and max tweak ink percentages minimizing weighted sum ofboth color errors:

$\begin{matrix}{{{{\min\limits_{\alpha_{\min},\alpha_{\max}}E_{CM}} + {p \cdot E_{WM}}} = {{\Delta\;{E_{76}\left( {R_{S},\frac{R_{\max} + R_{\min}}{2}} \right)}^{2}} + {{p \cdot \Delta}\;{E_{WM}\left( {R_{\min},R_{\max}} \right)}^{2}}}},\mspace{20mu}{s.t.\mspace{14mu}\begin{matrix}{{{R_{\max}(660)} - {R_{\min}(660)}} \geq \Delta_{660}} \\{{0 \leq \alpha_{\min}},{\alpha_{\max} \leq 1}}\end{matrix}}} & (9)\end{matrix}$where R_(max) and R_(min) correspond to Neugebauer spectral reflectancefrom Eq. (2) obtained for (spot and CMY) ink percentages α_(max) andα_(min), respectively. R_(S) refers to spectral reflectance of theoriginal spot color printed on a substrate. Color difference metricsΔE₇₆ and ΔE_(WM) are discussed above. Both metrics can be configured toreturn scalar values weighted by a constant penalty term p. In general,weight p is dependent on the color. From experiments conducted withprofessional designers, we now prefer to set the default value of theweight factor to p=1. Of course, varying p may result in additional orless signal detection robustness.

By formulating data hiding as an optimization problem, other printingpress or design-related constraints can be put in place. For example,designers may not allow a spot color ink to be screened due to physicalpress reasons, or may limit the amount of screening. By including thespot ink in α_(min) and α_(max) without any constraint will allow thespot ink to be modulated by the watermark tile. For example, a specificspot ink is moved from the fixed-ink path in FIG. 7 to the optimizationpath and is included in the optimization process along with CMY inks.Min and max tweaks then may include a spot ink component and thewatermark tile can be embedded by interpolating between these two colorsas described previously for the CMY case.

The optimization problem in Eq. (9) can be solved numerically with,e.g., the IPOPT library, using the underlying technology detailed in A.Wachter and L. T. Biegler, “On the implementation of a primal-dualinterior point filter line search algorithm for large-scale nonlinearprogramming.” Mathematical Programming, Vol. 106, issue (1): pages25-57, 2006, which is hereby incorporated herein by reference in itsentirety. IPOPT code is available as open source, e.g., athttp://www.coin-or.org/Ipopt.

Such an optimization can be carried out for a given image area, e.g., aspot color area have the same color values. Additionally, theoptimization can be carried out for each image area including differentcolor values. This might include optimizing an entire image or imagearea on a per pixel basis, or on an area by area basis.

FIG. 11 provides a graphical framework to consider some aspects of ourcolor embedding optimization technology, e.g., as previously discussedrelative to FIG. 7. SC1 represents a first color. To generalize anoptimization system, there can be at least 3 components: i) systemspace, ii) variables to optimize, and iii) system constraints. In oneimplementation the space may include a color gamut defined by a firstcolor channel (SC channel) and four process colors CMYK. Of course, thespace may include additional color channels, e.g., two or more spotcolors. In this implementation, the variables to optimize may include,e.g., eight (8) or more variables, e.g., SC2, CMY tint and ACMYK. Theconstraints may include, e.g., bounding features, ink properties,printer properties, use, package form factor (e.g., 3D model), detectioncriteria, performance standards and/or system limitations.

Returning to FIG. 11, SC1 could be a 100% value of a certain spot color.Unmodified, there is insufficient head room for SC1 to accommodate ahidden signal. So an approximation of SC1 can be achieved by movingalong both the SC channel and combining with other color channels. Theother colors can include, e.g., process color inks (CMYK). A distance(e.g., a color error metric) 160 is preferably minimized to achieve anapproximation of SC1 using an SC color channel value, SC2, +somecombination of the other color channels (e.g., CMYK). SC2 can be ascreened version of SC1.

Tweak values 162, which are introduced in the other color channels tocarry an information signal, are determined. A floor 166 can be setwithin an optimization function to maintain a particular robustness ofthe hidden data. For example, reflection at a certain spectral band canbe considered. Other robustness factors may include expected printdistortion (e.g., plate mis-registration), scanner noise, color screenproperties, printer resolution, illumination considerations, imagecharacteristics, color values, etc. The magnitude of the tweaks can beoptimized to ensure desired robustness. A visibility ceiling 164 canalso be set to establish a visibility constraints. Factors here mayinclude ink gamut limits, ink properties (e.g., metal effect),appearance model outputs, image masking outputs, printing angles, imagescharacteristics, HVS outputs, CSF outputs, etc. Such robustness factorsand visibility factors may be used as constraints for an optimizationfunction. There may be situations where robustness is key, so distancefunction 160 is less important relative to the floor established by 166.In other cases, visibility concerns my trump robustness causing theceiling 164 to contract.

Spot color and process color embedding can be implemented in many forms.Our preferred approach utilizes a software application plugin thatcooperates with digital imaging software such as Adobe Photoshop orAdobe Illustrator. The plugin can be crafted (e.g., using the AdobePhotoshop SDK or the Adobe Illustrator SDK and programming tools such asMicrosoft's Visual Studio) to provide user interfaces to select areaswithin digital image files for data hiding. The plugin may include orcall various functions, routines and/or libraries to perform the datahiding techniques disclosed herein, e.g., including optimizationprocesses, e.g., the IPOPT libraries, information signal generation(e.g., watermark embedder libraries), etc. The user interfaces may allowa user to select a type of data hiding for different digital imageareas, e.g., spot color embedding, process color tints, etc. The plugincan be constructed to provide user interfaces to accept parameters suchas robustness requirements, visibility requirements, global signal gain,etc. Such parameters can be entered graphically by moveable scale,entering numerical values, setting relative settings, etc. The plugincan be configured to operate autonomously. For example, the plugin canscan a digital version of a package design, determine flat areas (e.g.,spot colors), process color and white spaces. The plugin can run anoptimization to determine process color equivalents and tweaks for thespot colors, and determine CMY(K) tints for any white spaces or textareas.

An example plugin user interface is shown in FIG. 24. The interface canprovide various tunable options and displays, e.g., names of spot colorsin a particular digital image, spot color screen amount, CMY tintvalues, payload information (and error correction checksum data), embedinitiation tab, predicted detection maps (“LGS map,” which is a heat mapsimulation how detectable an embedded signal is to a POS scanner), atweak penalty to adjust global visibility of an embedded signal, a colormatch penalty to the adjust color hue or value due to watermark signal.In many cases the plugin is automated as discussed above, and can beconfigured to provide a reporting screen without (or prior to) userintervention.

Instead of a plugin in, the operations and functions described hereincan be directly incorporated into digital image software applications orstandalone applications.

Another implementation utilizes a web or cloud-based service. The webservice provides user interfaces to upload or create digital imagerycorresponding to product packaging. The web or cloud-based servicehouses or calls libraries, programs, functions and/or routines toachieve the spot color and process color embedding, includingoptimization, described herein.

The image processing operations for embedding and optimization may beimplemented as instructions stored in a memory and executed in aprogrammable computer (including both software and firmwareinstructions) or executed on one or more processors, implemented asdigital logic circuitry in a special purpose digital circuit, orcombination of instructions executed in one or more processors anddigital logic circuit modules. The methods and processes described abovemay be implemented in programs executed from a system's memory (acomputer readable medium, such as an electronic, optical or magneticstorage device). The methods, instructions and circuitry may operate onelectronic signals, or signals in other electromagnetic forms. Thesesignals further represent physical signals like image signals, inkvalues and percentages, as well as other physical signal types capturedin sensors. These electromagnetic signal representations are transformedto different states as detailed above to alter or modify ink values forphysical product packaging.

This formulation of the embedding problem is not limited to spot inkbeing overprinted by CMY inks. The same formulation can be used with anyset of inks that could be overprinted in the package design. Forexample, two spot colors could be used to embed information signals.This technique can be used for watermarking spot colors in ExtendedGamut printing processes, e.g., such as Hexachrome printing. Constraintsrelated to additional grayscale conversion weights from Eq. (7) can alsobe added to consider signal strength as seen by full-color devices suchas mobile phones.

III. Additional Implementations and Description

Other implementations, description and embodiments are provided below.

One alternative but related embedding technology uses a blend modeltaking, e.g., a 4 color SWOP profile, and creates a 5 color profile (4SWOP colors+S1 a) to create a 5 color search space. The search space canbe searched to find an optimized solution of robustness, readability,and minimized visibility changes. (Even if a black color is not used, itcan be advantageous to search across a 4 color space.)

A SWOP profile refers to a profile provided by or following aspecification of the “Specifications for Web Offset Publications.” TheSWOP specification covers many areas related to print production,complementing, extending and limiting those in other industry standards.The specification includes (but is not limited to) the following: I) Aspecification for the colors of the Cyan (C), Magenta (M), Yellow (Y)and key (Black) inks used in CMYK printing. Inks conforming to thespecification can be called SWOP inks. The specifications make referenceto, but are not identical to, the ISO standard ISO 2846-1:2006. II) Aspecification for the colors of proofs produced by various technologies,so they are close representation of the SWOP inks eventually used toprint. Proofs made from systems that meet these specifications may becalled SWOP Proofs. III) Specifications for expected dot gain (caused byink dots enlarging over absorbent papers). IV) Requirements forproducing halftones and color separation. V) Design constraints, such asthe minimum size of type which is to be printed reversed or knocked outof a background, to keep legibility.

A first approximation of a combined color (e.g., S1 a+CMY) may use thefollowing process:

1. a) Reduce spot color (S1) percentage to yield a screened back spotcolor (S1 a). This can aid in watermark detectability by a POS scanner,and b) estimate process color percentages (e.g., a CMY combination tooverlay the spot color).

2. Estimate colorimetric coefficients for composite color, e.g., % S1a+xC+yM+zY, where % is the spot color screening percentage, and x, y andz are weighting or percentage coefficients for their respective processcolors.

3. Correct color coefficients for spot overprint.

4. Determine values for overprint and percent spot color.

Predicting an actual color of a spot color ink when it is overprintedwith another ink(s), or vice versa, can characterize each colorindividually and predict the color of overprinting solids and halftonesby linearly combining the reflectance of all colors. Improvements can bemade to this prediction by selectively weighting the combined colors.See, e.g., Deshpande, K. and Green, P. “A simplified method ofpredicting the colorimetry of spot colour overprints,” 18th ColorImaging Conference: Color Science and Engineering Systems, Technologiesand Applications, pg. 213-216, San Antonio, USA 2010, which is herebyincorporated herein by reference in its entirety.

FIG. 13 is a flow diagram illustrating one implementation of colorblending to support digital watermarking with spot colors and processtints.

A spot color analysis starts with evaluating spot color S1. Spot colorS1 can be represented in terms of its approximate Lab values, e.g.,graphics software including Adobe Illustrator may include Lab librariesrepresenting various spot colors. Ink manufactures will also likely haveLab values associated with each spot color. Once Lab values areobtained, the values can be converted to CMYK equivalents. Look uptables, data sheets, transformation equations and/or libraries can beconsulted for this conversation. Of course, if CMYK values areoriginally available, one may be able to skip the Lab to CMYKconversion. It is then determined whether the Cyan component in the CMYKequivalent is less than or equal to 75%. If not, the spot color S1 isscreened back (e.g., using dot gain correction to the Lab values) untilthe Cyan component is less than or equal to 75%.

Let's take a moment to discuss the focus on Cyan. Recall from above thatwe are contemplating use of a POS scanner with a red LED (or laser)which peaks at or around 660 nm. Cyan (like Black) has very lowreflectivity at or around 660 nm.

Couple that with the spectral response of a RED LED scanner we wouldprefer to introduce watermark tweaks in the Cyan channel so they can bereadily ‘seen’ with a red scanner/camera. Such a Red LED capture deviceis likely monochromatic. Thus, the capture device (e.g., camera) only‘sees’ colors which reflect at or around 660 nm. If color stronglyreflects at this wavelength the camera ‘sees’ white. Bright yellow,magenta, pink, orange and white are all ‘seen’ as white by the capturedevice. If color reflects 0% at this wavelength (e.g., absorbs thewavelength) the camera ‘sees’ black. Dark blue, Cyan, green, purple andblack are all ‘seen’ as black by the camera.

Thus, when using a RED LED scanner, watermark detection includes aspectral dependence; successful watermark embedding, therefore, includesembedding receptive to the particular spectral dependence.

We left the FIG. 13 flow diagram discussion at screening back spot colorS1 if the CMYK equivalent is not less than or equal to 75%. Recall thatwe are going to combine the screen backed spot color S1 a with processcolor equivalents, with the watermark signal preferably being carried inthe process colors. And, if using a red color laser or LED, we want tomatch the red with tweaks in Cyan so that they can be more readily seenby the capture device. So, if the spot color S1 a is Cyan heavy, it mayrisk washing out the printed CMY. That is, the Cyan heavy S1 aintroduces noise such that the watermark tweaks in the underlyingprocess colors are difficult to detect. Depending on the application andtolerance for noise, a Cyan trigger in the range of, e.g., 60-85%, canbe used to decide whether to screen spot color S1.

The combined screened spot color+modulated process colors can beevaluated against the 100% spot to determine whether the combined tinthas an acceptable luminance error. For example, ΔE76, ΔE94 and/or ΔE2000values can be calculated. If a combined tint shows a large error, thendifferent watermark signal tweaks can be iteratively explored until andacceptable error is found. Acceptable in this context can bepredetermined based on use. For example, if detection robustness is aprimary concern, more watermark visibility can be tolerated.

Next, watermark tweaks are calculated for the CYM process colors. Thetweaks can be represented as, e.g., magnitude changes to the determinedprocess color percentage values. Once the tweaks are calculated, theycan be used to selectively transform the process colors to convey thedigital watermark signal. In some examples, determined tweaks areconverted to linear RGB, and scaled for underlying spot reflectivity inlinear RGB. These scaled values are converted back to CMY as embeddingmagnitudes or weights for magnitudes.

We have discussed a spectral dependency when reading watermarking with,e.g., a red LED capture device/camera. But using such a narrow bandillumination leaves a lot of watermark signal unusable by a detector.Recall from above that watermark tweaks in cyan (and yellow) are offsetwith magenta changes having opposite polarity. This helps reducewatermark visibility by keeping luminance changes at a minimum. Forexample, a monochrome perspective (e.g., an ink view) of a Cyan plane, aMagenta plane and a Yellow plane are shown with relative magnitude tweakchanges in FIG. 14. When the C, M and Y planes are superimposed in print(top left patch), luminance change attributable to the watermarkingtweaks is reduced.

But, if captured with a red LED scanner/camera, only the cyan tweaks areseen for watermark reading purposes. Watermark signal per unitvisibility can be increased by using, e.g., 2 or more colorilluminations. For example, with reference to FIG. 15, relative signalstandard deviation is increased (see middle bar) when illuminating withboth a red and blue LED (with a monochrome sensor), and furtherincreases when illuminating with a white light LED and capturing withRGB sensors (right bar). Cyan plane is seen by red LED scanner/sensor,magenta by green LED scanner/sensor and yellow by blue LEDscanner/sensor.

With reference to FIG. 16, we show relative timing with 2 colorillumination with a monochromic sensor(s). Illumination with Red andGreen LED may allow for better capture and timing. For example, if usingonly 1 monochromic sensor, the illumination of the red and green LEDscan be delayed or spaced to allow for image data corresponding to Redillumination to be captured and buffered before image data correspondingto Green illumination is captured on the same sensor. Data capturedcorresponding to red illumination can be combined with data capturedcorresponding to green illumination to bolster the hiddensignal-to-noise ratio.

In other arrangements, e.g., 3 color illumination and multiplemonochromatic sensors, each sensor includes its own particular colorfilter. For example, each sensor includes a particular filter so that itcan see Cyan, Magenta or Yellow. Information from these sensors can becombined to further increase signal strength prior to embedding.

Flexo prints are sometimes used for plastics and foils, including thoseused in the food industry. This type of printing often is difficult whentrying to introduce fine ink percentage changes (e.g., forwatermarking), or to achieve close plate color registration. This typeof printing typically uses spot colors, and typically not process colorinks due to the large screen size.

Some of the above implementations utilize process colors+screened spotcolor, with a watermark signal conveyed by modulating the processcolors. Since flexo printing does not typically include process colors,a different approach can be employed.

One such approach combines a flexo spot color, perhaps even a screenedversion of such, and combines with an additional spot color which ispreferably light, has high reflectance at all wavelengths except at oraround 660 nm, and potentially another area. FIGS. 17, 18 and 19 showsome potential candidates for the additional spot color including cyans,greens, and purples. The original spot color is modulated with tweaks toconvey a watermark signal, which can have a luminance offset from theadditional spot color.

Some criteria for selecting suitable overprint spot colors may include:

(if using a red LED scanner) reflectance between 50%-80% at 660 nm; CIEL* between 82-90; and are a representative color in every CIE hue 18degree increments/20 colors total in first investigation.

Another approach pairs spot colors. For example, given a spot color, two(2) different spot colors which can be each modulated to include adigital watermark signal are identified. The modulated 2 different spotcolors when combined are visually a close approximation of the originalspot color.

With reference to FIG. 36, still another data hiding technology involvesspot color substitution. For example, instead of screening back a spotcolor as discussed above, a first spot color S_(a) can be replaced orsubstituted with a second spot color S_(b)+an overprinted CMY(K) tint.Selecting a second spot color S_(b) avoids changing a workflow operationto include a color screen. Sometimes controlling a screen can be tricky,including being able to obtain a precise amount of screen. Some printerslack the ability to precisely screen colors and inks. To avoidscreening, the substitute second spot color S_(b) can be printed at100%, and then a CMY(K) overprinted tint added. The overprinted CMY(K)tint preferably conveys a digital watermark signal. The overprintedsecond spot color S_(b)+CMY(K) tint is preferably a close approximationto 100% of the first spot color S_(a).

The substitution process begins by selecting one or more substitute spotcolors. For example, a database including spot color information (and,e.g., corresponding Lab, CMYK and/or RGB information) can be consultedto determine a set of candidates. Candidate selection may involvefinding a set of “close” spot colors relative to the first spot colorS_(a). Close may be determined, e.g., by color distances metrics, basedon corresponding Lab, CMYK or RGB values, between the first spot colorS_(a) and candidate substitute spot colors. A resulting set ofcandidates preferably includes 2 or more substitute spot colors, e.g.,2-12 spot colors (e.g., referred to as S_(b1)-S_(b12)).

An exhaustive search may be carried out over an entire spot colorlibrary to find close candidates. For example, the 2014 version ofPANTONE+ coated color book includes 1,755 spot colors, and if one ofthem is the first spot color S_(a), then the other 1,754 can beevaluated relative to S_(a) (e.g., a distance metric or color errormetric for each of the 1,754 spot colors relative to S_(a)). Theshortest distance or lowest error metric spot colors can be included inthe set of candidates. (Since the PANTONE color book is not very wellorganized (e.g., similar colors are not always labeled with consecutiveindices), an exhaustive search can likely find potential closecandidates.

For a given spot color library, the search space can be limited prior tocarrying out an exhaustive search. For example, printer gamut,reflectance criteria, etc. can be used to limit or prune the searchspace. The candidate search can be carried out against this limited orpruned search space.

Other constraints can be optionally considered when selectingcandidates. For example, only those spot colors with a higherreflectance relative to cyan (C) at a predetermined wavelength (e.g., ator around 660 nm) can be considered as viable spot color substitutes.

As another optional constraint, after creating a short list foralternate spot colors (say, e.g., 3 spot colors B, C and D areidentified as potential substitutes to an original spot color A), onlythose with relatively higher luminance are selected. For example, thetop third or half (or top 2-5) candidates in terms of luminance aremaintained. Considering luminance may help keep colors vivid and avoidmaking them dull after swapping out the original spot color A.

Once a set of candidates (e.g., S_(b1)-S_(b12)) is selected, acorresponding CMY(K) tint can be determined for each spot color S_(bi),where i is an integer, within the set of candidate spot colors (e.g.,S_(b1)-S_(b12)). For example, a table or database can be consulted tofind the corresponding CMY color percentages or weightings correspondingto the first spot color S_(a). These values can be used as the tint. Inanother alternative, for example, and with reference to FIG. 7, acandidate spot color S_(bi) can be used instead of the 75% screened spotcolor (2). A color match error E_(CM) between the S_(bi)+tint and thefirst spot color S_(a) can be minimized in determining an optimal CMYtint for that particular S_(bi). In alternative implementations, theoptimization process discussed above with respect to Eq. 9 can becarried out with each S_(bi) (instead of a spot color screen) to find anoptimized E_(CM) and E_(WM) (watermark error). The resulting CMY tint asmodulated by the watermark tile can be used in further evaluation.

Once a CMY tint is selected for each S_(bi), one (1) or more finalcandidates are selected. For example, a digital simulation of theS_(bi)+overprinted CMY tint can be analyzed and compared to the firstspot color S_(a). Final candidates may include those with the smallestLab distance or Chroma distance. For the Lab distance, and for (L₁*,a₁*, b₁*) and (L₂*, a₂*, b₂*), two colors in L*a*b*:

ΔE_(ab)*=√{square root over ((L₂*−L₁*)²+(a₂*−a₁*)²+(b₂*−b₁*)²)}. Achronia distance may look similar, but without the first (L₂−L₁)² term.Of course, other distance metrics can be used, e.g., ΔE94, ΔE2000.

The final candidates can be provided through a user interface forconsideration by a designer for selection as a substitute of the firstspot color S_(a). In some cases the best 1-4 final candidates asdetermined by a Lab distance metric and the best 1-4 final candidates asdetermined by a Chroma distance metric are all provided to the designer.In other implementations, the closest matching S_(bi)+tint (in terms ofsmallest distance values relative to S_(a)) can be automaticallyselected and used as a substitute. As an optional constraint, only thoseS_(bi)+tint candidates with a smaller reflectance relative to S_(a) at apredetermined wavelength (e.g., at or around 660 nm) can be selected asa final candidate.

Once a S_(bi)+CMY tint is selected, the CMY tint can be modulated with awatermark tile so as to carry a watermark signal. The S_(bi)+modulatedtint can be then printed, e.g., on product packaging.

A related process to determine substitute or alternative spot colors isdiscussed with reference to FIG. 37.

1. In step 31, it is determined how much an original spot color S_(a)should be screened back. For example, and with reference to FIG. 7, wecan again look to the optimization problem of Eq. 9 where one of theoptimization solutions can be an amount or percentage of the spot colorS_(a) screen. The result is a spot color screen S_(s) that can beevaluated relative to substitute candidates. Recall from FIG. 7 that wemay seek to minimize color match error (e.g., between a 100% originalspot color and a screened original spot color+a CMY(K) tint) andwatermark error. Other constraints may include signal strength error anda spectral dependence (e.g., at or around 660 nm).

2. In step 32, the Pantone spot color universe is examined to find 1-i(where i is an integer) candidate spot color substitutes S_(bi) having:i) a low color error (or shortest distance) between the candidatesubstitute spot color S_(bi) and the screened back version S_(s) of theoriginal spot color, and also ii) a candidate S_(bi) with a color valuethat is brighter than the screened backed version S_(s) of the originalspot color. Color error or color distance metrics can be determinedusing, e.g., Lab distance, Chroma distance, ΔE94, ΔE2000 or CIEDE2000,etc. “Low” can be determined relative to a predetermined threshold valueor by a relative evaluation, e.g., the “lowest” 1-5 substitute spotcolors are selected for further evaluation. The second prong, a colorvalue that is brighter than the screened back version S_(s), can beviewed from a scanner's perspective, e.g., what is the substitute spotcolor's spectral reflectance at or around 660 nm. Generally, the biggerthe brightness value is, the brighter the color is. In step 32, if theoriginal spot color is brighter than paper white (e.g., like theflorescent colors such as Pantone 804, 805 and 806) or other threshold,the process can be optionally configured to not enforce the 2^(nd) prongconstraint of “brighter than” when searching for substitute spot colorcandidates.

3. In step 33, choose the top candidate S_(b1) from the list ofcandidates S_(bi) (from step 32) and determine a CMY tint such that the100% S_(b1) (i.e., not screened) plus (+) the overprinted CMY tint isclose to the 100% original spot color. “Closeness” can be determined byLab distance, Chroma distance, ΔE94, ΔE2000 or CIEDE2000, e.g., relativeto a predetermined target value (e.g., at or below a JND). Of course,the target value can be above a JND in other implementations. Thechanging variable here is the CMY tint, since we are using 100% S_(b1).Additionally, we prefer to determine color closeness for the CMY tintplus the selected substitute spot color S_(b1) only; that is, prior tomodulating the CMY tint to carry a watermark signal. In some case we caniteratively vary the CMY process color tint until the color error orcolor distance between i) a combination of a CMY color tint and thesubstitute spot color candidate Sb1, and ii) the spot color Sa, isminimized. In other cases, we using an optimization function to minimizeerror or distance.

4. In step 34, and after the amount of CMY tint has been decided, awatermark signal (e.g., with the watermark tile in FIG. 7) is used tomodulate the CMY tint, while maintaining S_(b1) at 100% (i.e., notscreened). Sb1 and the watermarked CMY tint are combined (or they can becombined at printing).

5. As an optional step 35, repeat steps 32-34 for additional substitutespot color candidates S_(b2)-S_(bi). While this step is optional,practice has shown that designers like to make choices.

6. In step 35, test or proof prints are printed, which includes at leastone substitute spot color candidate S_(b1) with its watermarked CMYtint. A final candidate can selected or approved after a visualinspection. The CMY tint and substitute spot color candidate S_(b1) canbe combined at printing or beforehand.

As an optional step, steps 32 and 33 are carried out while simulatingprinting with different substrates. Thus, the closeness determinationtakes into account the S_(b1), CMY tint and substrate relative to theoriginal spot color.

Added text on packaging can sometimes interfere with a watermark signal.For example, black text may been seen by a red LED camera as black (highabsorption). Colors with high reflectance at the target peak scannerresponse can instead be used for text. Referring again to FIG. 19,orange is an ideal color for text.

Oftentimes printed packaging is applied to non-flat surfaces. Forexample, printed plastic foil can be shrink-wrapped on and around acontainer. Examples include, e.g., yogurt cups, energy drink bottles,toppings containers, etc. The plastic foil can be modeled to thecontainer with shrink wrapping, e.g., heat wrapped. Heat wrappingintroduces distortion to the printing. Distortion is modeled for printedfoil intended for a yogurt cup in FIGS. 20a and 20b . Item B is aprinted plastic foil pre-shrink wrap. Item A includes distortionmodeling of the post shrink wrapped version of item B. In FIG. 20b , atrapezoid distortion is shown by the gridding.

One method for modeling trapezoidal distortion determines circumferencepoints for the top and bottom of the yogurt cup. A linear transformationmaps points from the top (wider) circumference into the bottom (smaller)circumference points, with transformation distortion depending on atarget 3D container's shape. Of course, other 3D models can be used toestimate or predict how a watermark signal will be mapped onto a 3Dobject like product packaging. Such transformations can be used topre-condition host images. For example, various 3D models are discussed,e.g., in U.S. Pat. No. 8,570,343, which is hereby incorporated herein byreference in its entirety, can be used.

Such distortion will adversely affect watermark detection. For example,a watermark may include an orientation component that can be comparedagainst a reference template to help determine distortion of capturedimagery including the orientation component. The shrink wrap process canfurther complicate the interpretation of the orientation and distortion.

One method addresses distortion by warping (e.g., transforming) a hostimage prior to watermark embedding. For example, if item B is a hostimage, then a transformation T1 which models expected distortion isapplied to item B to yield item A. A watermark signal is embedded inimage A, and then image A is inversely transformed to yield item B. Theinversely transformed item B, which includes a distorted watermarksignal, can then be shrink-wrapped or otherwise applied to the container(yogurt cup). The shrink wrapping introduces distortions estimated forby the transformation T1, which yields a watermark signal which is moreclosely aligned to the originally embedded watermark (e.g., as embeddedin image A).

We designed a test to investigate whether watermark detection robustnessis better if:

-   -   Image transformed T1 and then watermarked=>A    -   Image watermarked and then Transformed T2=>B

In this test we simulated a marked yogurt cup moving parallel to a redLED scanner bed about ¼″ above a bottom scanner to simulate a checkerscanning the yogurt cup. The cup was simulated by two different A & Bprinted graphic on a paper substrate, and then wrapped around a yogurtcontainer. The simulated cup is then passed in front of a verticalcamera at various speeds as shown in FIG. 21. The results are shown inFIG. 22, where percentage of successful watermark detections atdifferent watermark resolutions (50 & 70 watermark per inch) andincreasing scanning speeds/inch is shown. At each tested scanning speed,the watermark was detected with a higher percentage when embedded afterimage transformation (B).

In assignee's U.S. Provisional Patent Application Nos. 60/032,077, filedAug. 1, 2014, and 62/102,270, filed January 12, each of which is herebyincorporated herein by reference, we discussed various digitalwatermarking embedding Workflow Processes.

A related process may include one or more of the following processes,with general reference to FIG. 23:

1) Receive digital package files from, e.g., via a secure FTP.

2) Pre-Flight to determine that we have all info. Pre-flight is a termused to describe a preliminary step that evaluates received information,and may include reformatting, decompressing files, and an overallevaluation whether the received digital page files can be assembled intoa printable package. Package artwork is typically represented by acollection of files in a variety of different formats e.g., Bitmaps(*.tiff, *psd, etc.), vector imagery (*.ps, *.ai, etc.), and fonts(*.abf, *.ttf, etc.). A final rendered packaged can be “built” using theaforementioned files using a variety of different strategies, from a1-layer bitmap to numerous layers of vector and bitmap imagery utilizingmultiple fonts.

3) Enter Package/Retailer/Printer/Supplier in CRM system, e.g.,Microsoft Dynamics CRM (not shown). Optionally, the materials mayinclude an XML file which can be used to automatically enter theinformation. In this case, a manual check will help ensure accuracy.

4) Assign to Teams. For example, different tasks can be assigned todifferent work stations, or to available operators. An operator queuecan be examined to determine availability.

5) Create an identity file in an identity management system (e.g.,housed in the cloud) and associate the GTIN. The creation and managementof these services can be accomplished through a web-portal to theidentity management system or programmatically through Web APIs. If thepackaging materials includes a barcode number, e.g., in a GTIN format,this information can be obtained and provided as a watermark payload orpart of a watermark payload, or to a storage location at which awatermark will point to.

6) Review Files—Different Classifications. These classification mayinclude assignment of package embedding difficultly. This may promptadditional resources or billing requirements.

7) Print-out Initial Client Proof.

8) EMBED Digimarc Barcode. For example, the spot color and process colorembedding methods and technology disclosed herein can be employed atthis step.

-   -   8a) In the digital realm, grade the embedded Digimarc Barcode.        For example, watermark signal strength across the package can be        assigned values, and based on corresponding reads for the values        a grade can be assigned per package side or area.

9) Print Watermarked Proof

10) Test on POS Scanner. This is a preliminary test to see if the proofwill read.

11) Assemble Package for Manual Test

12) Manual Test. This can be a detailed process, where each package faceis tested, e.g., at different reading angles. For example, each side istested on a POS scanner with a vertical camera and a horizontal camera.The package is passed over the scanner, e.g., 2, 4 or 8 times per sideand then number of reads is recorded. The side is rotated, e.g., 90degrees and the process is repeated for that side, rotated again andretested, etc. Each package side can be so tested and the resultsrecorded. A grade can be assigned based on successful reads. Of course,the process is benefited from automation where a package is passed infront of a scanner, e.g., with a robot arm, conveyor belt or some othermovement mechanism.

13) Complete QC Checklist

-   -   13a) compare results of digital grade and manual grade; decide        whether to accept or refine embedded package.

14) Send Approved file to Customer via FTP

IV. Implementations of Adaptive Embedding Framework

One goal of a color visibility model is to create an objective visualdegradation model due to digital watermarking of an image. For example,a model may predict how noticeable or visible image changes will be dueto watermark insertion. Highly noticeable changes can be reduced ormodified to reduce watermark visibility, and/or to create equalwatermark visibility (or lack thereof) across an image. For example, anerror metric above or relative to the standard “Just NoticeableDifference” (JND) can be used to determine noticeable changes.

In a first implementation, with reference to FIG. 25, a digitalwatermarked image is compared to a processed version of an originalimage to determine a visibility map. The visibility map can be used toweight watermark embedding of the original image, e.g., to reducewatermark strength in high visibility areas. The process starts withconversion of an original image into the so-call CIELAB space, resultingin L*, a* and b* color representations. As mentioned above, the L*coordinate represents the perceived lightness or luminance, an L* valueof 0 indicates black and a value of 100 indicates white. The CIE a*coordinate position goes between “redness” (positive) and “greenness”(negative), while the CIE b* goes between “yellowness” (positive) and“blueness” (negative). The original image is digitally watermarked andthen converted into the CIELAB space. For example, the watermarking forthis initial process may use a uniform embedding strength across theentire image.

Contrast between the original image and the marked image can bedetermined, and then contrast sensitivity functions (CSFs) can beapplied to each of the L*, a* and b* channels. For example, the L* CSFsdiscussed in Daly, “Visible differences predictor: an algorithm for theassessment of image fidelity,” F. L. van Nes et al. “Spatial ModulationTransfer in the Human Eye,” J. Opt. Soc. Am., Vol. 57, Issue 3, pp.401-406 (1967), or Johnson et al, “On Contrast Sensitivity in an ImageDifference Model,” PICS 2002: Image Processing, Image Quality, ImageCapture Systems Conference, Portland, Oreg., April 2002; p. 18-23 (whichis herein incorporated herein in its entirety), can be used. In othercases a bandpass filter, with a drop off toward low-frequencies, can beapplied to the L*. The processed or blurred L* channel (from theoriginal image) can be used to determine visibility masking. Forexample, areas of high contrast, edges, features, high variance areas,can be identified for inclusion of more or less watermarking strength.Some areas (e.g., flat area, edges, etc.) can be entirely masked out toavoid watermarking all together.

For the a* and b* channels, chrominance CSFs can be applied to therespective channels, e.g., such CSFs as discussed in Johnson et al,“Darwinism of Color Image Difference Models;” G. J. C. van der Horst etal., “Spatiotemporal chromaticity discrimination,” J. Opt. Soc. Am.,59(11), 1482-1488, 1969; E. M. Granger et al., “Visual chromaticitymodulation transfer function,” J. Opt. Soc. Am., 63(9), 73-74, 1973; K.T. Mullen, “The contrast sensitivity of human colour vision to red-greenand blue-yellow chromatic gratings,” J. Physiol., 359, 381-400, 1985;each of which are hereby incorporated herein by reference in theirentirety. In other cases, a low-pass filter is used which has a lowercut-off frequency relative to the CSF of luminance.

Channel error difference can then be determined or calculated. Forexample, on a per pixel basis, L*, a* and b* data from the originalimage are compared to the blurred (e.g., processed with respective CSFs)L*, a* and b*channels from the watermarked image. One comparisonutilizes ΔE₇₆:

Using (L₁*, a₁*, b₁*) and (L₂*, a₂*, b₂*), colors in L*a*b*, the errorbetween two corresponding pixel values is:

ΔE_(nb)*=√{square root over ((L₂*−L₁*)²+(a₂*−a₁*)²+(b₂*−b₁*)²)}, whereΔE_(AB)*≈2.3 corresponds to a JND (just noticeable difference). Othercomparisons may utilize, e.g., ΔE₉₄ or ΔE2000.

Of course, and more preferably used, is an error determination for theblurred (CSF processed) L*a*b* from the original image and the CSFblurred L*a*b* from the watermarked image.

The output of the Calculate Channel Difference module identifies errormetrics. The error metrics can be used to identify image areas likely toinclude high visibility due to the inserted digital watermark signal. Wesometimes refer to this output as an “error map”. Typically, the lowerthe error, the less visible the watermark is at a particular area, imageblocks or even down to a signal pixel.

The visibility mask and the error map can be cooperatively utilized toguide digital watermarking. For example, watermark signal gain can bevaried locally according to the error map, and areas not conducive toreceive digital watermark, as identified in the visibility mask, canaltogether be avoided or receive a further signal reduction.

One limitation of the FIG. 25 model is that it does not take intoaccount local luminance influences for Contrast Sensitivity Functions(CSF), particularly for the a* and b* chrominance channels. Withreference to FIG. 26, we propose a color visibility model for use with adigital watermark embedder that seeks equal visibility across an imageby locally varying watermarking embedding strength based on predictedvisibility influenced, e.g., by local image luminance. A CSF for eachcolor channel can be varied spatially depending on the luminance of thelocal image content.

The luminance content of the original image provides potential maskingof changes due to watermarking in chrominance as well as luminance. Forexample, where a watermark signal comprises mostly high frequencycomponents, the masking potential of the original image is greater atregions with high frequency content. We observe that most high frequencycontent in a typical host image is in the luminance channel. Thus, theluminance content of the host is the dominant contributor to maskingpotential for luminance changes and chrominance changes for highfrequency components of the watermark signal.

Returning to FIG. 26, we may add several modules relative to the FIG. 25system, e.g., “Calculate Local Luminance” and “blur SCALED CSF” modules.The FIG. 26 visibility model system uses separate CSFs for contrastvariations in luminance and chrominance (red-green and blue-yellow)channels. The width, characteristics or curve of the CSF in each channelcan be scaled or modified depending on the luminance of the local imagecontent. For example, for a given pixel, local luminance in aneighborhood around the pixel can be evaluated to determine a localbrightness value. The local brightness value can be used to scale ormodified a CSF curve. The neighborhood may include, e.g., 4, 8 or morepixels. In some cases, the CSF is adjusted so that more blurring occursas the luminance of the local region decreases. The error differencebetween the contrast of the blurred (or unblurred) original and theblurred marked image can be measured using a color difference metric,e.g., ΔE76, ΔE94 or ΔE2000.

With reference to FIG. 27a , one objective may include embedding digitalwatermarking into images with equal visibility. That is, the imageincludes watermarking embedded therein at different signal strengthvalues to achieve uniform or equal visibility. During the embeddingstage, the visibility model can predict the visibility of the watermarksignal and then adjust the embedding strength. The result will be anembedded image with a uniform watermark signal visibility, with theembedding strength varying locally across the image depending oncharacteristics of the cover image's content. For example, a visibilitymap generated from the FIG. 26 system is used to reshape (e.g., locallyscale according to an error map and/or mask embedding or avoidance areasaccording to a visibility map) a watermark signal. The original signalis then embedded with the reshaped watermark signal to create an equalvisibility embedded (EVE) image. In such a case, the watermark signallocally varies to achieve an overall equal visibility.

Some visibility advantages of EVE vs. uniform strength embedding (USE)are shown in FIG. 27b . The visibility of the USE varies from area toarea, as see in the bottom left image. In comparison, when embedding thesame image area with EVE (bottom right image), the watermark visibilityappears equal. The bottom left and right images represent the same imagearea highlighted in blue in the upper right image.

An additional implementation involving CSFs modified to consider localluminance is discussed below.

In image fidelity measures, the CSF is commonly used as a linear filterto normalize spatial frequencies such that they have perceptually equalcontrast thresholds. This can be described by the following shiftinvariant convolution:

$\begin{matrix}{{{\overset{\sim}{f}\left( {x,y} \right)} = {{{h\left( {x,y} \right)}*{f\left( {x,y} \right)}} = {\sum\limits_{m}{\sum\limits_{n}{{h\left( {m,n} \right)}{f\left( {{x - m},{y - n}} \right)}}}}}},} & (10)\end{matrix}$where f(x,y) is an input image, h(x,y) is the spatial domain CSF, and{tilde over (f)}(x,y) is the frequency normalized output image.

For a luminance dependent CSF model, we allow the CSF to vary spatiallyaccording to the local luminance of the image, e.g.:

$\begin{matrix}{{\overset{\sim}{f}\left( {x,y} \right)} = {\sum\limits_{m}{\sum\limits_{n}{{h\left( {m,{n;x},y} \right)}{{f\left( {{x - m},{y - n}} \right)}.}}}}} & (11)\end{matrix}$

Since evaluating this shift variant convolution directly can becomputationally expensive, in some implementations we seek anapproximation that can be more computationally efficient. The use ofimage pyramids for fast image filtering is well-established. An imagepyramid can be constructed as a set of low-pass filtered anddown-sampled images f_(l)(x,y), typically defined recursively asfollows:

$\begin{matrix}{{f_{0}\left( {x,y} \right)} = {{f\left( {x,y} \right)}\mspace{14mu}{and}}} & (12) \\{{f_{l}\left( {x,y} \right)} = {\sum\limits_{m}{\sum\limits_{n}{{h_{0}\left( {m,n} \right)}{f_{l - 1}\left( {{{2x} - m},{{2y} - n}} \right)}}}}} & (13)\end{matrix}$for l>0 and generating kernel h₀(m,n). It is easily shown from thisdefinition that each level f_(l)(x,y) of an image pyramid can also beconstructed iteratively by convolving the input image with acorresponding effective kernel h_(l)(m,n) and down-sampling directly tothe resolution of the level, as follows:

$\begin{matrix}{{{f_{l}\left( {x,y} \right)} = {\sum\limits_{m}{\sum\limits_{n}{{h_{l}\left( {m,n} \right)}{f_{0}\left( {{{2^{l}x} - m},{{2^{l}y} - n}} \right)}}}}},} & (14)\end{matrix}$where h_(l)(m,n) is an l-repeated convolution of h₀(m,n) with itself.

For image filtering, the various levels of an image pyramid can be usedto construct basis images of a linear decomposition representing thepoint-spread response of the desired filtering, e.g.:

$\begin{matrix}{{{\overset{\sim}{f}\left( {x,y} \right)} = {\sum\limits_{l}{a_{l}{{\overset{\sim}{f}}_{l}\left( {x,y} \right)}}}},} & (15)\end{matrix}$where α_(l) is the coefficient of the basis function {tilde over(f)}_(l)(x,y) obtained by up-sampling the corresponding pyramid levelf_(l)(x,y) back to the base resolution.

We can use the effective convolution kernel h_(l)(x,y) as aninterpolating kernel, e.g.,

$\begin{matrix}{{{{\overset{\sim}{f}}_{l}\left( {x,y} \right)} = {\underset{m}{4^{l}\sum}{\sum\limits_{n}{{h_{l}\left( {{x - {2^{l}m}},{y - {2^{l}n}}} \right)}{f_{l}\left( {m,n} \right)}}}}},} & (16)\end{matrix}$such that each basis function {tilde over (f)}_(l)(x,y) can be describedby a simple shift-invariant convolution of the input image with acomposite kernel {tilde over (h)}_(l)(x,y):{tilde over (f)} _(l)(x,y)={tilde over (h)} _(l)(x,y)*f _(l)(x,y),  (17)where {tilde over (h)}_(l)(x,y)=h_(l)(x,y)*h_(l)(x,y). Thus, consideringEq. (15), we assert that the optimal representation is obtained byminimizing the sum of the squared error between the desired CSF and theGaussian representation; e.g.,

$\begin{matrix}{{a = {\arg\mspace{11mu}{\min\limits_{a}E}}},{where}} & (18) \\{{E = {\sum\limits_{x}{\sum\limits_{y}\left( {{h\left( {x,y} \right)} - {\sum\limits_{l}{a_{l}{{\overset{\sim}{h}}_{l}\left( {x,y} \right)}}}} \right)^{2}}}},} & (19)\end{matrix}$and a=[α₁, α₂, . . . ]. A linear least-squares problem, which can besolved using software packages such as, e.g., Matlab® or GNU Octave, canbe utilized to solve equation 18. Further, the optimization can bepre-calculated for each local luminance of interest and stored in alook-up table, noting that for one example application each coefficientα_(l) is spatially varying according to the local luminance levelL_(f)=L_(f)(x,y) of f(x,y), i.e., α_(l)=a_(l)(L_(f))=a_(l)(L_(f)(x,y)).

While the development of our approach has been conducted for a basisimage at the resolution of an input image, our methods can be conductedwithin a multi-resolution scheme, reducing the calculation of thespatially variant convolution into a pyramid reconstruction withspatially variant analysis coefficients.

Results and examples varying a CSF in each channel depending on theluminance of the local image content is described in Appendix B,included as part of this specification, and which is hereby incorporatedherein by reference in its entirety.

The following documents are hereby incorporated herein by reference:Lyons, et al. “Geometric chrominance watermark embed for spot color,”Proc. Of SPIE, vol. 8664, Imaging and Printing in a Web 2.0 World IV,2013; Zhang et al. “A spatial extension of CIELAB for digitalcolor-image reproduction” Journal of the Society for Information Display5.1 (1997): 61-63; Van Nes et al. “Spatial modulation transfer in thehuman eye,” Journal of Optical Society of America, vol. 57, issue 3, pp.401-406, 1967; Van der Horst et al. “Spatiotemporal chromaticitydiscrimination,” Journal of Optical Society of America, vol. 59, issue11, 1969; and Watson, “DCTune,” Society for information display digestof technical papers XXIV, pp. 946-949, 1993.

In some cases, even better results can be achieved by combining anattention model with our above visibility model when embeddingwatermarks in color image data. An attention model generally predictswhere the human eye is drawn to when viewing an image. For example, theeye may seek out flesh tone colors and sharp contrast areas. One exampleattention model is described in Itti et al., “A Model of Saliency-BasedVisual Attention for Rapid Scene Analysis,” IEEE TRANSACTIONS ON PATTERNANALYSIS AND MACHINE INTELLIGENCE, VOL. 20, NO. 11, November 1998, pgs.1254-1259, which is hereby incorporated herein by reference. High visualtraffic areas identified by the attention model, which would otherwisebe embedded with a relatively strong or equal watermark signal, can beavoided or minimized by a digital watermark embedder.

Disclosure from Appendix B is provided below:

Full-Color Visibility Model Using CSF which Varies Spatially with LocalLuminance

Abstract:

A full color visibility model has been developed that uses separatecontrast sensitivity functions (CSFs) for contrast variations inluminance and chrominance (red-green and blue-yellow) channels. Thewidth of the CSF in each channel is varied spatially depending on theluminance of the local image content. The CSF is adjusted so that moreblurring occurs as the luminance of the local region decreases. Thedifference between the contrast of the blurred original and marked imageis measured using a color difference metric.

This spatially varying CSF performed better than a fixed CSF in thevisibility model, approximating subjective measurements of a set of testcolor patches ranked by human observers for watermark visibility. Theeffect of using the CIEDE2000 color difference metric compared toCIEDE1976 (i.e., a Euclidean distance in CIELAB) was also compared.

Introduction

A full color visibility model is a powerful tool to measure thevisibility of the image watermark. Image watermarking is a techniquethat covertly embeds additional information in a cover image, such thatthe ownership, copyright and other details about the cover image can becommunicated. Watermarks used for packaging are inserted in thechrominance domain to obtain the best robustness per unit visibility.See Robert Lyons, Alastair Reed and John Stach, “Geometric chrominancewatermark embed for spot color,” Proc. Of SPIE, vol. 8664, Imaging andPrinting in a Web 2.0 World IV, 2013. The chrominance image watermark isembedded in a way that the color component in the cover image isminimally altered and is hardly noticeable, due to human vision system'slow sensitivity to color changes.

This visibility model is similar to Spatial CIELAB (S-CIELAB). SeeXuemei Zhang and Brian A. Wandell, “A spatial extension of CIELAB fordigital color-image reproduction” Journal of the Society for InformationDisplay 5.1 (1997): 61-63. The accuracy of this model was tested bycomparing it to subjective tests on a set of watermarked color patches.The model was found to significantly overestimate the visibility of somedark color patches. A correction was applied to the model for thevariation of the human contrast sensitivity function (CSF) withluminance as described below. After luminance correction, goodcorrelation was obtained with the subjective tests.

The luminance and chrominance CSF of the human visual system has beenmeasured for various retinal illumination levels. The luminance CSFvariation was measured by Floris L. Van Nes and Maarten Bouman, “Spatialmodulation transfer in the human eye,” Journal of Optical Society ofAmerica, vol. 57, issue 3, pp. 401-406, 1967 and the chrominance CSFvariation by G J Van der Horst and Maarten Bouman, “Spatiotemporalchromaticity discrimination,” Journal of Optical Society of America,vol. 59, issue 11, 1969. These measurements show a variation in peaksensitivity of about a factor of 8 for luminance and 5 for chrominanceover retinal illumination levels which change by about a factor of 100.

Since the retinal illumination can change by about a factor of 100between the lightest to darkest area on a page, the CSF peak sensitivityand shape can change significantly. The function is estimated by theaverage local luminance on the page, and a spatially dependent CSF isapplied to the image. This correction is similar to the luminancemasking in adaptive image dependent compression. See G J Van der Horstand Maarten Bouman, “Spatiotemporal chromaticity discrimination,”Journal of Optical Society of America, vol. 59, issue 11, 1969.

The luminance dependent CSF performed better than a fixed CSF in thevisibility model, when compared to subjective measurements of a set oftest color patches ranked by human observers for watermark visibility.Results of our model with and without luminance correction are comparedto S-CIELAB in Section 2, Visual Model Comparison. The method ofapplying a spatially dependent CSF which depends on local imageluminance is described in Section 3, Pyramid Processing Method.

The visibility model is then used to embed watermark into images withequal visibility. During the embedding stage, the visibility model canpredict the visibility of the watermark signal and then adjust theembedding strength. The result will be an embedded image with a uniformwatermark signal visibility, with the embedding strength varyingdepending on the cover image's content. This method was compared to auniform strength embed in terms of both visibility and robustness, andthe results are shown in Section 4, Watermark Equal Visibility Embed.

Visual Model Comparison

Psychophysical Experiment

To test the full-color visibility model a psychophysical experiment wasconducted. The percept of degradation caused by the watermark wascompared to the results of the visibility model, as well as to theS-CIELAB metric.

A set of observers were asked to rate their perception of the imagedegradation of 20 color patch samples using a quality ruler. The qualityruler (illustrated in [FIG. 13a ]) increases in watermark strength fromleft (B) to right (F). The color samples were viewed one at a time at aviewing distance of approximately 12 inches. The samples were presentedusing the Latin square design (see Geoffrey Keppel and Thomas Wickens,“Design and analysis: A researcher's handbook.” Prentice Hall, pp.381-386, 2004) to ensure a unique viewing order for each observer.

See [FIG. 28a ] Quality ruler increasing in degradation from B (slight)to F (strong).

All 22 participants passed the Ishihara color test. There were eightfemale and 14 male participants, with an average age of 43. Theirprofessions and experience varied. Four people had never participated ina visibility experiment, 12 had some experience and six had participatedon several occasions.

Thumbnails of the 20 color patches are illustrated in See [FIG. 28b ].The color samples were chosen largely based on the results of a previousexperiment; where it was observed that the visibility model haddifficulty accurately predicting the observer response with darker colorpatches. Additionally, one color patch had a much higher perceived andpredicted degradation. Ten of the original samples were included in thesecond experiment. Dark patches, patches which were expected to have ahigher perception of degradation and memory colors were added tocomplete the set of 20 patches. The experiment and the quality rulerpatches were all printed with an Epson Stylus 4880 on Epson professionalphoto semi-gloss 16 inch paper.

See [FIG. 28b ] Thumbnails of the 20 color patch samples with thewatermark applied.

The mean observer scores for the 20 color samples are plotted in See[FIG. 29]. In general the colors on the far right are lighter. Asdiscussed in the previous experiment, the cyan1 patch was observed tohave a higher level of degradation. In this second experiment, othercolors with similar properties were determined to have a similarly highperception of degradation.

See [FIG. 29] The mean observer responses with 95% confidence intervals.

Validation of the Visibility Model

The motivation for the psychophysical experiment is to test how well theproposed full-color visibility model correlates to the perception of thedegradation caused by the watermark signal. The model without and withthe luminance adjustment are plotted in See [FIG. 30] and See [FIG. 31],respectively.

See [FIG. 15] Mean observer response compared with the proposedvisibility model. The solid black line is the polynomial trendline.

See [FIG. 16] Mean observer response compared with the proposedvisibility model with luminance adjustment.

The addition of the luminance adjustment primarily affected the darkercolor patches, darkgreen, foliage and darkblue1. CIEDE94 and CIEDE2000color difference models were also considered, however there was not aclear advantage to using the more complex formulas.

See [FIG. 32] Mean observer response compared with S-CIELAB.

The S-CIELAB values are also plotted against the mean observer responseSee [FIG. 32].

Two different methods were used to compare the different metrics to theobserver data, Pearson's correlation and the coefficient ofdetermination (R²). Both correlation techniques describe therelationship between the metric and observer scores. The coefficientindicates the relationship between two variables on a scale of +/−1, thecloser the values are to 1 the stronger the correlation is between theobjective metric and subjective observer results. The correlations aresummarized in Table 1.

TABLE 1 Pearson and R² correlation between the observers' mean responsesand the objective metrics. For both tests, the proposed full-colorvisibility model with the luminance adjustment shows the highestcorrelation. Visibility model using CIE ΔE₇₆ No Adjust With AdjustS-CIELAB Pearson 0.81 0.86 0.61 R² 0.70 0.85 0.38

As shown in Table 1, all three objective methods have a positivecorrelation to the subjective results with both correlation methods. Thefull-color visibility model with the luminance adjustment had thehighest correlation with both the Pearson and R² correlation tests,while S-CIELAB had the lowest.

Pyramid Processing Method

In image fidelity measures, the CSF is commonly used as a linear filterto normalize spatial frequencies such that they have perceptually equalcontrast thresholds. This can be described by the following shiftinvariant convolution:

$\begin{matrix}{{{\overset{\sim}{f}\left( {x,y} \right)} = {{{h\left( {x,y} \right)}*{f\left( {x,y} \right)}} = {\sum\limits_{m}{\sum\limits_{n}{{h\left( {m,n} \right)}{f\left( {{x - m},{y - n}} \right)}}}}}},} & \left( {1a} \right)\end{matrix}$where f(x,y) is an input image, h(x,y) is the spatial domain CSF, and{tilde over (f)}(x,y) is the frequency normalized output image.

For our luminance dependent CSF model, we allow the CSF to varyspatially according to the local luminance of the image, i.e.:

$\begin{matrix}{{\overset{\sim}{f}\left( {x,y} \right)} = {\sum\limits_{m}{\sum\limits_{n}{{h\left( {m,{n;x},y} \right)}{{f\left( {{x - m},{y - n}} \right)}.}}}}} & \left( {2a} \right)\end{matrix}$

Since evaluating this shift variant convolution directly can becomputationally expensive, we seek an approximation that is moreefficient.

The use of image pyramids for fast image filtering is well-established.An image pyramid can be constructed as a set of low-pass filtered anddown-sampled images f_(l)(x,y), typically defined recursively asfollows:

$\begin{matrix}{{{f_{0}\left( {x,y} \right)} = {f\left( {x,y} \right)}}{and}} & \left( {3a} \right) \\{{f_{l}\left( {x,y} \right)} = {\sum\limits_{m}{\sum\limits_{n}{{h_{0}\left( {m,n} \right)}{f_{l - 1}\left( {{{2x} - m},{{2y} - n}} \right)}}}}} & \left( {4a} \right)\end{matrix}$for l>0 and generating kernel h₀ (m,n). It is easily shown from thisdefinition that each level f_(l)(x,y) of an image pyramid can also beconstructed iteratively by convolving the input image with acorresponding effective kernel h_(l)(m,n) and down-sampling directly tothe resolution of the level, as follows:

$\begin{matrix}{{{f_{l}\left( {x,y} \right)} = {\sum\limits_{m}{\sum\limits_{n}{{h_{l}\left( {m,n} \right)}{f_{0}\left( {{{2^{l}x} - m},{{2^{l}y} - n}} \right)}}}}},} & \left( {5a} \right)\end{matrix}$where h_(l)(m,n) is an l-repeated convolution of h₀(m,n) with itself.

For image filtering, the various levels of an image pyramid are used toconstruct basis images of a linear decomposition representing thepoint-spread response of the desired filtering, i.e.:

$\begin{matrix}{{{\overset{\sim}{f}\left( {x,y} \right)} = {\sum\limits_{l}{a_{l}{{\overset{\sim}{f}}_{l}\left( {x,y} \right)}}}},} & \left( {6a} \right)\end{matrix}$where α_(l) is the coefficient of the basis function {tilde over(f)}_(l)(x,y) obtained by up-sampling the corresponding pyramid levelf_(l)(x,y) back to the base resolution.

We use the effective convolution kernel h_(l)(x,y) as an interpolatingkernel, i.e.,

$\begin{matrix}{{{{\overset{\sim}{f}}_{l}\left( {x,y} \right)} = {4^{l}{\sum\limits_{m}{\sum\limits_{n}{{h_{l}\left( {{x - {2^{l}m}},{y - {2^{l}n}}} \right)}{f_{l}\left( {m,n} \right)}}}}}},} & \left( {7a} \right)\end{matrix}$such that each basis function {tilde over (f)}_(l)(x,y) can be describedby a simple shift-invariant convolution of the input image with acomposite kernel {tilde over (h)}_(l)(x,y):{tilde over (f)} _(l)(x,y)={tilde over (h)} _(l)(x,y)*f(x,y),  (8a)where {tilde over (h)}_(l)(x,y)=h_(l)(x,y)*h_(l)(x,y). Thus, consideringEq. (6a), we assert that the optimal representation is obtained byminimizing the sum of the squared error between the desired CSF and theGaussian representation; i.e.,

$\begin{matrix}{{a = {\arg\mspace{11mu}{\min\limits_{a}E}}},{where}} & \left( {8a} \right) \\{{E = {\sum\limits_{x}{\sum\limits_{y}\left( {{h\left( {x,y} \right)} - {\sum\limits_{l}{a_{l}{{\overset{\sim}{h}}_{l}\left( {x,y} \right)}}}} \right)^{2}}}},} & \left( {9a} \right)\end{matrix}$and a=[α₁, α₂, . . . ]. This is a standard linear least-squares problemand can be solved using standard software packages, like Matlab® or GNUOctave. Further, the optimization can be pre-calculated for each localluminance of interest and stored in a look-up table, noting that for ourapplication each coefficient α_(l) is spatially varying according to thelocal luminance level L_(f)=L_(f)(x,y) of f(x,y), i.e.,α_(l)=a_(l)(L_(f))=a_(l)(L_(f)(x,y)).

While the development of our approach has been conducted for basis imageat the resolution of the input image, the procedure can be conductedwithin a multi-resolution scheme, reducing the calculation of thespatially variant convolution in Eq. (3.2) into a pyramid reconstructionwith spatially variant analysis coefficients.

Watermark Equal Visibility Embed

[FIG. 33] shows an example from a cover image mimicking a packagedesign. The design has two embedding schemes: on the left the watermarksignal strength is uniform across the whole image, and on the right thewatermark signal strength is adjusted based on the prediction from thevisibility model. Since the human visual system is approximately a peakerror detector, the image degradation caused by the watermark signal isdetermined by the most noticeable area. In this example, the hilly areain the background has the most noticeable degradation, as shown in themagnified insets. The visibility model is used to find this severedegradation. The signal strength in this area is reduced which improvesthe overall visibility of the embedded image, making it more acceptable.The total watermark signal on the right is 40% more than that on theleft, but visually, the marked image on the right is preferable to theleft one, because the degradation in the most noticeable area is reducedsignificantly.

[FIG. 34] shows the calculated visibility for the uniform signalstrength embedding (left) and the visibility model adjusted embedding(right). Notice that the visibility map is smoother on the right than onthe left.

In terms of watermark detection, the embedding scheme with visibilitymodel based adjustment can accommodate more watermark signal withoutcreating a very noticeable degradation, thus making the detection morerobust. To demonstrate the powerfulness of applying the visibilitymodel, we performed a stress test with captures of 4 images from the twoembedding schemes at various distances and perspectives. The other 3images from the uniform visibility embedding are shown in See [FIG. 35].Their visibility maps are not included but instead the standarddeviation of each visibility map is listed in Table 2. The percentage ofsuccessful detection is shown in Table 3.

These two tables show that the equal visibility embedding showed asignificant visibility improvement over the uniform strength embeddingscheme, together with robustness that was about the same or better.

See [FIG. 33] Watermark embedding with uniform signal strength (left)and equal visibility from the visibility model (right). The insets aremagnified to show image detail.

See [FIG. 34] Visibility map from uniform signal strength embedding(left) and equal visibility embedding (right).

See [FIG. 35] Apple tart, Giraffe stack and Pizza puff design used intests.

TABLE 2 Standard deviation of the visibility maps on the 4 images fromthe two embedding schemes. Test image Uniform strength embedding Equalvisibility embedding Granola 18.32 9.71 Apple Tart 8.19 4.96 GiraffeStack 16.89 11.91 Pizza Puff 11.81 8.27

TABLE 3 Detection rate on 4 images from the two embedding schemes, outof 1000 captures each image/embedding. Test image Uniform strengthembedding Equal visibility embedding Granola 18% 47% Apple Tart 50% 58%Giraffe Stack 47% 49% Pizza Puff 63% 61%

CONCLUSIONS

A full color visibility model has been developed which has goodcorrelation to subjective visibility tests for color patches degradedwith a watermark. The best correlation was achieved with a model thatapplied a luminance correction to the CSF.

The model was applied during the watermark embed process, using apyramid based method, to obtain equal visibility. Better robustness andvisibility was obtained with equal visibility embed than uniformstrength embed. Additional Reference: Andrew Watson, “DCTune,” Societyfor information display digest of technical papers XXIV, pp. 946-949,1993.

CONCLUDING REMARKS

Having described and illustrated the principles of the technology withreference to specific implementations, it will be recognized that thetechnology can be implemented in many other, different, forms. Toprovide a comprehensive disclosure without unduly lengthening thespecification, applicant hereby incorporates by reference each of theabove referenced patent documents in its entirety. Appendix A and B areexpressly included as part of this specification and are incorporatedherein by reference in their entirety.

The modules, methods, processes, components, technology, apparatus andsystems described above may be implemented in hardware, software or acombination of hardware and software. For example, the visibility modelsystems (e.g., FIGS. 25, 26 and 27), spot color and process colorembedding and optimizations may be implemented in software, firmware,hardware, combinations of software, firmware and hardware, aprogrammable computer, electronic processing circuitry, digital signalprocessors (DSP), graphic processing units (GPUs), a programmablecomputer, electronic processing circuitry, and/or by executing softwareor instructions with a processor, parallel processors, multi-coreprocessor and/or other multi-processor configurations.

The methods and processes described above (e.g., watermark embedders anddetectors) also may be implemented in software programs (e.g., writtenin C, C++, C#, R, Assembly, Objective-C, Shell, Scheme, Scratch, MATLAB,Visual Basic, Java, Python, Tcl, Perl, Scheme, Ruby, executable binaryfiles, etc.) stored in memory (e.g., a computer readable medium, such asan electronic, optical or magnetic storage device) and executed by oneor more processors, multi-core processors, distributed systems (orelectronic processing circuitry, hardware, digital circuit, etc.).

The particular combinations of elements and features in theabove-detailed embodiments (including Appendices A & B) are exemplaryonly; the interchanging and substitution of these teachings with otherteachings in this and the incorporated-by-reference patents anddocuments are also contemplated.

What is claimed is:
 1. A method for generating a color design for aprinted article comprising: obtaining data associated with a first spotcolor S_(a), the first spot color S_(a) comprising a component of acolor design; determining a plurality of substitute spot colorcandidates S_(b1)-S_(bi), where i is an integer, by evaluating—for eachsubstitute spot color candidate—color distance metrics between dataassociated with the substitute candidate spot color and the dataassociated with the first spot color S_(a); determining a Cyan (C),Magenta (M) and Yellow (Y) tint for each of the plurality of substitutespot color candidates S_(b1)-S_(bi); using one or more electronicprocessors, simulating an overprint of each of the plurality ofsubstitute spot color candidates S_(b1)-S_(bi) with its respective CMYtint, and for each of the overprinted substitute spot color candidates,generating an Lab or Chroma distance metric relative to the first spotcolor S_(a); based on generated distance metrics, determining final spotcolor candidates; for at least one of the final spot color candidates,and using one or more electronic processors, transforming its respectiveCMY tint with a digital watermark signal; and substituting the at leastone of the final spot color candidates plus its transformed CMY tint forthe first spot color S_(a) in the color design.
 2. The method of claim 1in which the data associated with the substitute candidate spot colorcomprises Lab data.
 3. The method of claim 1 in which the dataassociated with the substitute candidate spot color comprises RGB orCMYK data.
 4. The method of claim 1 in which said determining final spotcolor candidates comprises determining Lab distance values relative tothe first spot color S_(a) for each of the plurality of substitute spotcolor candidates S_(b1)-S_(bi).
 5. The method of claim 1 in which saiddetermining final spot color candidates comprises determining Chromadistance values relative to the first spot color S_(a) for each of theplurality of substitute spot color candidates S_(b1)-S_(bi).
 6. A systemfor encoding information for a printed article comprising: memory forstoring i) data representing a first color (S1), ii) data representingsecond color data (S2), and iii) data representing third color data(S3), in which encoded information is provided by modulating the datarepresenting third color data (S3) with max (positive) and min(negative) tweaks; one or more processors configured for: determining acolor error between S1 and a combination of S2 and S3, the combinationincluding the max and min tweaks; determining an information modulatingerror associated with the max and min tweaks; and optimizing thecombination of S2 and S3, including minimizing the color error andminimizing the information modulating error; and an output for providingthe optimized combination of S2 and S3 for use in a design to be printedon the printed article.
 7. The system of claim 6 in which the optimizingincludes determining color weights and a global signal strength, thecolor weights to be applied to the data representing third color data(S3) and the global signal strength for regulating the modulating. 8.The system of claim 7 in which the optimizing is constrained by spectralreflectance between 630 nm to 680 nm.
 9. The system of claim 7 in whichthe optimizing is constrained by spectral reflectance between 655 nm to670 nm.
 10. The system of claim 7 in which the data representing thirdcolor data (S3) comprises data representing two (2) or more processcolors, and the data representing second color data (S2) represents asecond color.
 11. The system of claim 10 in which the second colorcomprises a screened-back version of the first color.
 12. The system ofclaim 6 in which the max and min tweaks correspond to a 2-dimensionalimage digital watermark tile.
 13. The system of claim 6 furthercomprising a display, in which said one or more processors areconfigured for providing a graphical user interface through which a usercan select embedding options.
 14. A system for transforming color datato include an information signal embedded therein, comprising: memorystoring spot color data, and system constraint information, the systemconstraint information including at least one spectral dependency; andone or more processors configured as an optimizer bounded by the systemconstraint information, the optimizer generates C (Cyan), M (Magenta), Y(Yellow) process color data to be combined with a screen of the spotcolor data to yield a minimized color error approximation of the spotcolor data, the optimizer generates tweak values in terms of at leastΔC, ΔM, ΔY for modulating the C, M, Y process color data to carry theinformation signal, the tweak values optimized based on an error metric;and one or more processors configured as an embedder that transforms theC, M, Y process color data with the tweak values.
 15. A method forgenerating a design for a printed article comprising: minimizing: i) acolor match error between a first color (S1) and a combination of datarepresenting second color data (S2) and data representing third colordata (S3), the combination including auxiliary encoded information inthe data representing third color data, and ii) an informationmodulating error associated with modulations of the data representingthird color data (S3) that carry the auxiliary encoded information, saidminimizing being constrained by a spectral component associated with ananticipated information detector, in which said minimizing yields colorpercentage values associated with at least the data representing thirdcolor data (S3) and a signal strength value for regulating themodulations; and outputting the color percentage values and signalstrength value for use in a design to be printed on the printed article.16. The method of claim 15 in which spectral component comprises ametric associated with a spectral reflectance between 630 nm to 680 nm.17. The method of claim 15 in which the modulations correspond to a2-dimensional image digital watermark tile.